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EXECUTIVE SUMMARY 


Report of the Panel on Large Scale Computing 
in Science'and Engineering 


Large scale computing is a vital component of science, engineering, and 
modern technology, especially those branches related to defense, energy, 
and aerospace. In the 1950’s and 1960’s the U. S. Government placed high 
priority on large scale computing. The United States became, and continues 
to be, the world leader in the use, development, and marketing of 
"supercomputers," the machines that make large scale computing possible. 

In the 1970’s the U. S. Government slackened its support, while other 
countries increased theirs. Today there is a distinct danger that the U.S. 
will fail to take full advantage of this leadership position and make the 
needed investments to secure it for the future. 

Two problems stand out: 

Access . Important segments of the research and defense communities lack 
effective access to supercomputers; and students are neither familiar 
with their special capabilities nor trained in their use. 

Access to supercomputers is inadequate in all disciplines. Agencies 
supporting some disciplines such as fusion energy, atmospheric sciences, 
and aerodynamics have funded National computing facilities through which 
their remote users have limited networking capabilities. In those 
disciplines that attempt to fund computing through individual research 
grants, access to large scale computing remains minimal. 

Future Supercomputers . The capacity of today's supercomputers is 
several orders of magnitude too small for problems of current urgency 
in science, engineering, and technology. Nevertheless, the development 
of supercomputers, as now planned in the U.S., will yield only a small 
fraction of the capability and capacity thought to be technically 
achievable in this decade. 

Significant new research and development effort is necessary to overcome 
technological barriers to the creation of a generation of supercomputers 
that tests these technical limits. Computer manufacturers in the U. S. 
have neither the financial resources nor the conmercial motivation in the 
present market to undertake the requisite exploratory research and 
development without partnership with government and universities. 

Unless these barriers are overcome, the primacy of U. S. science, 
engineering, and technology could be threatened relative to that of other 
countries v/ith national efforts in supercomputer access and development. 
Although the Federal Government is the first and by far the largest 
customer for supercomputers, there are no national plans to stimulate the 
development and use of advanced computer technology in the U. S. 



Reconmendations: 


The Panel reconmends the establishment of a National Program to stimulate 
exploratory development and expanded use of advanced computer technology. 
The Program has four principal components, each having short- and long-term 
aspects. Underlying than all is the establishment of a system of effective 
computer networks that joins government, industrial, and diversity 
scientists and engineers. The technology for building networks that allow 
scientists to share facilities and results is already developed and 
understood; no time should be lost in connecting existing research groups 
and computing facilities. * 

The four components of the recommended program are: 

1. Increased access for the scientific and engineering research 
community through high bandwidth networks to adequate and 
regularly updated supercomputing facilities and experimental 
computers; 

2. Increased research in computational mathematics, software, 
and algorithms necessary to the effective and efficient use 
of supercomputer systems; 

3. Training of personnel in scientific and engineering 
computing; and 

4. Research and development- basic to the design and 
implementation of new supercomputer systems of substantially 
increased capability and capacity, beyond that likely to 
arise from conmercial requirements alone. 

The Panel recommends that this program be coordinated within the Federal 
Government by an interagency policy committee, and that an 
interdisciplinary Large Scale Computing Advisory Panel be established to 
assist in its planning, implementation, and operation. 

The Panel believes that current funding levels are insufficient to maintain 
the Nation’s leadership in large scale computing. Federal agencies that 
depend on large scale computing to fulfill their missions must work 
together to reexamine priorities and to create a coherent program 
responsive to their individual missions. The Panel has set forth policy 
and planning issues and has outlined some options for implementation. 
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I. Introduction 


Supercomputers are the fastest and most powerful scientific computing 
systems 1 available at any given time: they offer speed and capacity, or 
special characteristics, significantly greater than on^the most widely 
available machines built primarily for conmercial use. Large scale 
scientific computing is the application of supercomputers to the solution 
of a model or simulation of a scientific or engineering problem through the 
appropriate use of nunerical algorithms and techniques. 

The availability of supercomputers during the past thirty years has been 
crucial to the Nation's advances in science, engineering, national 
security, and industrial productivity. Supercomputers have been essential 
to scientific and engineering investigations in areas such as atmospheric 
research, astrophysics, molecular biology, integrated circuit design, and 
fusion research. The weapons programs of DOE, cryptographic analysis, and 
weather forecasting are dependent on the availability of computational 
facilities. The use of supercomputers in the aerospace, petroleum, 
semiconductor, and nuclear industries contributes substantially to the 
nation's productivity. The development of supercomputers has significant 
spinoffs for all the technologically based components of the national 
economy. Research and development in semiconductor.technology and in 
computer research has directly supported and expanded the defense, 
industrial, medical, and consumer segments of the economy. 

The U.S. is the acknowledged leader in the development and use of 
supercomputers. In 1970 this Nation was preeminent in all aspects ef 
electronic, computer, and computational technology. However, America's 
present leadership in supercomputers is challenged in the areas of 
components, development, and applications. Recently, Hitachi_has begun 
marketing what is claimed to be the first 16k bipolar ECL RAM , this 
device, representative of the continuing advances of Japanese 
microelectronic manufacturers, is designed for applications in today's 
scientific computers. Fujitsu, Nippon Electric Company, and Hitachi have 
each developed supercomputers, which are claimed to compare favorably with 
the available, or announced, American systems. American universities and 
research centers, which have historically created new applications of 


^ These include specialized machines, such as array processors, that are 
equal to or, for some problems, more powerful than general purpose 
2 mainframes. 

The term supercomputer, as used in this report, encompasses hardware, 
software, supporting peripherals, and the facilities and personnel needed 
for their appropriate use. Appendix I reproduces three papers presented 
to the Panel: NCAR Computing Capabilities and Services , by W. Macintyre; 
Magnetic Fusion Energy and Computers , by J. Killeen; and The Potential of 
Los Alanos National Laboratory to Provide Large Scale Computational 
Capabilities to the Research Community , by B. Buzbee and D. Sparks. 

These papers present descriptions of supercomputer facilities and of 
their access from remote facilities through networking. 

^ See Electronic Engineering Times, December 6 , 1982. 


-3- 









supercomputers and sustained research in computational mathematics, have 
lagged behind their Japanese and European counterparts in the installation 
of supercomputers. 

Significant national thrusts in supercomputing are being pursued by the 
governments of Japan, West Germany, France, and Great Britain. Some of 
these, notably the Japanese effort, center on the development of 
supercomputers; others, on the provision of supercomputers, or access to 
them through networks, to the scientific and engineering research 
conmunity. The British program, for example, is designed to provide 
research scientists and engineers in academic and government laboratories 
access to supercomputers through modern workstations connected to a 
high-speed national network. These agressive foreign national initiatives 
provide a striking contrast to the current state of planning in the U.S.. 
The domestic computer industry continues its vigorous research and 
development efforts in the supercomputer field; however, it is felt that 
these efforts, necessarily dictated by commercial conditions, are less than 
they could be and far less than should be for the national scientific and 
technical capability as a whole. The U.S. research community does not have 
sufficient access to supercomputing facilities, as is documented in 
nunerous studies, papers, and reports directed toward specific disciplines 
and specific agencies. A partial bibliography of these studies is included 
in this report. 

Expressions of concern that the U.S. is failing to exploit its position of 
leadership in supercomputing are being voiced from many quarters. 

Reflecting this concern, the NSF/DOD Coordinating Comnittee requested, in 
April of 1982, that a workshop be organized to explore the problems * needs, 
and opportunities in large scale computing. This Workshop, sponsored by 
NSF and DOD with the cooperation of DOE and NASA, was led by a panel of 
fifteen scientists and engineers from a broad spectrum of disciplines. It 
took place at the NSF on June 21-22, 1982, and was attended by over one 
hundred participants. Experts in the use, design, and management of large 
scale computers from the computing, defense, and other industries, 
government laboratories, universities, and research funding agencies were 
included. The lists of the participants in this Workshop are contained in 
the Supplement. 

The Panel assessed the role of supercomputing in scientific and engineering 
research; surveyed the current use, availability, and adequacy of 
supercomputers*} and considered near- and long-term needs. Subsequent to 
the June 21-22 Workshop, nunerous meetings of smaller groups of 
participants have taken place; in particular, experts on computer 
development (Group 3 of the Lax Panel) met at Bellaire, Michigan, on August 
23-24, 1982, to further explore avenues for assuring the development of 
future supercomputers. From these meetings a large nunber of suggestions 
and position papers have been directed to the Panel and to the Organizing 
Comnittee. This report is an attempt, on the part of the Organizing 
Comnittee and the Panel, to outline both the results of the Workshop and 
the subsequent discussions and contributions. The Panel has chosen not to 
repeat all the detailed technical arguments or examples of the use of 
supercomputers found in the literature. A bibliography and appendices are 
included. 
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O/erall, this report outlines the issues and options for the U.S. to 
maintain its leadership in supercomputers and supercomputing. Because the 
issues involve many Federal agencies, government laboratories, 
universities, private sector companies, and scientific disciplines, they 
need to be addressed on a National basis and require Federal study, 
planning, and support. The Panel's report attempts to bring the 
undanental issues to the attention of policymakers; however, it 
deliberately avoids details of an organizational, programmatic, or 
budgetary nature. 


II Summary of Findings and Recommendations 
Sunmary of Findings 

Large scalj| computing is a vital component of science, engineering and 
technology , bringing together theory and applications. It is essential 
for the design of many technologically sophisticated products, and is 
making possible for the first time the analysis of very complex scientific 
and engineering problems which to date have defied analjS^nd 

bSlnTnoSd t SlSi^ eS ' EXan ’ PleS ° f thS im P° rtance of superccmputing are 

Renormalization group techniques 6 are a megor theoretical breakthrough 
miiL OVlde \ n f. w frarnework for the understanding of a number of 

and engineerin 6 problems ranging from problems in 
devei™ ^® ldt J eo [ y . th e onset of phase transitions in materials, the 
PvnfoffT 1 of „turbulence, propagation of cracks in metals, and the 
nroh?ii atl ° n K f 01 J reservoirs * Only a minute fraction of these 
tpchni G3 h 6 ^ olved anal ytically. Large scale computational 

5 ave een essent i a l to the use of renormalization group 
6Ven ^°? ay ' s largest computational machines are not P 
sufficiently powerful to address most of these problems'. 

air r ?on a wi L th d S g i n USl H g 3 S perC ° mputer has resuited in the design of an 

tecSiLe^ + rag th f devel °P ed b y previous experimental 

q eo .. The solution of the Navier-Stokes equations with sufficient 


’ fo ^ exarn Pi e >J & D for National Strength . Center for Strategic and 
International Studies, Georgetown University, Washington, D.C. 1982. 

5 ^ ee a - 1 - so T he Defense Science Board Summer St udy. 1981 ’ ’ 

f u Pf lement °f this Report, H. B. Keller and J.*R. Rice describe in 
some detail scientific and engineering areas in need of supercomputers 

nr?hi n li X 11 contai Pf a nLrn her of examples of scientific and engineering 
p oblems successfully addressed on supercomputers as well as additional 

6 JEFf £ e ? uiring supercomputing capabilities not yet available. 

These techniques were devised to handle the description of phenomena with 
interactions spanning an extremely wide scale. K. G Wilson 
the 1982 Nobel Prize in Physics for his contribution; to the toeor elToTl 
evelopment and application of renormalization group techniques to 
critical phenomena. See his contributions to this report in the 

7 Supplement, as well as in Appendices II and III. 

8 Z ee ’ x *} Appendix II, several problems posed by K. G. Wilson. 

^oT^ 10 " “* ' ,,VendS 3nd P30ing ItOTS in mutational 


-5- 







resolution to represent faithfully fluid behavior became practical with 
the current (Class VI) generation of supercomputers. The wings of the 
Boeing 767 and of the European Airbus 310 were designed by computational 
methods on such machines, resulting in this most significant improvement. 

The aerodynamic design of an entire aircraft is not feasible with today's 
supercomputers; it is estimated that machines 100 times more powerful are 
needed for this purpose. The design of jet engines, involving the 
simulation of complex three-dimensional fluid flows and associated 
chemical reactions, also requires significantly increased computational 
capability and capacity. 

In design, especially of advanced weapons systems, large scale 
computational modeling is an essential substitute for experimentation. 
Similarly, the design of future generations of nuclear power plants, 
and their operation—relying on real-time simulation for their 
control—require computational facilities several orders of magnitude 
greater than those available today. 

Perhaps the most significant applications of scientific computing lie 
not in the solution of old problems but in the discovery of new 
phenomena through numerical experimentation; the discovery of 
nonergotic behavior, such as the formation of solitons, and the 
presence of strange attractors as universal features common to a large 
class of nonlinear systems are examples of this scientific process. 

Current and feasible supercomputers are extremely powerful scientific and 
engineering tools. They permit the solution of previously intractable 
problems, and motivate scientists and engineers to explore and formulate 
new areas of investigation. They will surely find significant applications 
not yet imagined. For these reasons, the Panel believes that it is in the 
National interest that access to constantly updated supercomputing 
facilities be provided to scientific and engineering researchers, and that 
a large and imaginative user community be trained in their uses and 
capabilities. 

The U.S. has been and continues to be the leader in supercomputer 
technology and in the use of supercomputers in science and engineering. 

The present position of leadership is evidenced by the dominance of the 
supercomputer market by American producers and by the successful 
exploitation of supercomputing at national laboratories. However, the 
Panel finds that this position of leadership is seriously undermined by the 
lack of broad scale exploration, outside of a few national laboratories, of 
the scientific and engineering opportunities offered by supercomputing, and 
by a slowdown in the introduction of new generations of supercomputers. 

This threat becomes real in light of the major thrust in advanced 
supercomputer design that is being mounted by the Japanese Government and 
industry, and by vigorous governmental programs in the United Kingdom, West 
Germany, France, and Japan to make supercomputers available and easily 
accessible to their research and technological ccranunities. 

American preeminence in large scale computing has been a result of the 
confluence of three factors: the vitality of the U.S. computer industry, 
the far-sighted policies of the Federal government, and the leadership of 
scientists and engineers from universities and government laboratories. 

The Atomic Energy Commission, on the urging of John von Neunann, initiated 
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the use of large scale computation in research and weapons design; NASA, 
prodded by Hans Mark, advanced the use of supercomputing in its scientific 
programs. American universities and government laboratories conducted the 
research that formed the basis for constructing and applying computers, 
trained the needed scientific and engineering personnel, and made computers 
and computing an essential tool in scientific and engineering research. 

The Federal government vigorously implemented policies that supported these 
efforts, granted generous funds for computation, and, through its role as 
the major purchaser of scientific computers, provided the incentives and 
insured the market for these unique machines. Forward-looking corporations 
exploited the scientific and engineering opportunities, developed an 
advanced industrial technology, and created this most vital component of 
the American economy. 

During the 1970's the Federal government retreated fhom its support of 
large scale computing in universities. The NSF program to provide and 
expand university computing facilities for scientific and engineering 
research was terminated in 1972; at about the same time IBM discontinued 
its generous discounts for the purchase of computing equipment by academic 
institutions as a result of pressures frcm the Justice Department and 
competitors. Since then large scale university computing facilities have 
withered while the action shifted to national laboratories and to 
industrial users. The most advanced scientific computer of the early 
seventies, the CDC 7600, was not installed on a single American canpus, 
although it was available to researchers at several foreign universities 
and research institutes. 

This continues today. With the exception of two universities and a few 
government laboratories, either dedicated to special tasks or specific 
disciplines, universities and government research installations lack 
supercomputers. 


Within the Government, fully integrated supercomputer facilities are found 
exclusively at dedicated national laboratories such as Los Alanos National 
Laboratory (LANL) and Lawrence Livermore National Laboratory (LLNL) in 
support of weapons and fusion programs; and NASA installations, the 
Geophysical Fluid Dynamics Laboratory, and the National Center for 
Atmospheric research (NCAR) in support of aerospace, oceanographic, and 
atmospheric research programs. The National Magnetic Fusion Energy 
Computer Center (NMFECC) of LLNL is accessible in interactive mode to 
researchers at remote locations by a high speed network. 10 On the other 
hand, the National Bureau of Standards and most DOD laboratories do not 
have supercomputers and far too few universities have the specialized 
computational equipment needed for scientific computing (e.g., array 
processors). As a result of limited access to supercomputing facilities by 
the broad research conmunity, significant research opportunities 1T have 


g 

See, in Appendix I, Partial Inventory and Announced Orders of Class VI 
IQ Machines . ' " 

n See Appendix I for a description of some of these facilities. 

See, for example, the "Prospectus for Computational Physics," [2], 
"Future Trends in Condensed Matter Theory and the Role and Support of 
Computing" [4], "Report by the Subcomnittee on Computational 
Capabilities for Nuclear Theory," [28], and "An Assessment of 
Computational Resources Required for Ocean Circulation Modeling," [35], 
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been missed, and the younger generation of researchers is inadequately 
trained in large scale computing. 

The need.for access to large scale computational facilities has become so 
critical that several universities, assuming significant financial risks, 
have felt it essential to acquire supercomputers. Several more are 
seriously considering doing so, and others are in the process of forming 
consortia for this purpose. Some of these endeavors have applied for 
Federal funding without which they may have financial difficulties. Other 
groups are pressing funding"agencies to expand or replicate highly 
successful facilities, such as these at NCAR, NMFECC, and NASA-Anes, at 
universities or at national laboratories. Class VI scienti fic re mote 
computing services are available from a few~conmerclal service bureaus, but 
neither the .academic nor the government resear ch comm unities make extensiv e 
of this re mu ree l T his seems due to a ~cCTn5Trratt0ff~of lack of fundsT& r 
computing .services, the perceived high cost associated with these service s, 
and a lack of sophisticated high-speed networking facilities" It is an~ 
indication oflihe absence of a national plan that a substantial number of 
leading scientists are clamoring for access to supercomputers at the same 
time that some supercomputing facilities are underutilized. 

A supercomputer is a general purpose scientific instrument serving a broad 
and diverse base of users. The decline of supercomputing at universities 
is analogous to the decline of instrumentation; neither large scale 
computing nor instrumentation can be sustained in a stable manner through 
funding of individual researc^grants, where their costs must compete with 
that of scientific personnel. 

The findings of the Panel regarding the development of supercomputers are 
as alarming as the findings on their access and availability. The U.S. 
supercomputing market is, at this time, doninated by Cray Research (CRAY-1) 
and Control Data Corporation (CYBER 205). The Japanese vendors, Hitachi 
(S-210/20) and Fujitsu (VP-200), have announced the delivery of 
supercomputers in the near future, and these machines appear to be 
comparable to the available American systems. The Japanese are striving to 
become serious competitors of domestic manufacturers, and U.S. dominance of 
the supercomputer market may soon be a thing of the jpgst. The Japanese 
Government-sponsored National Super Computer Project is aimed at the 
development, by 1989, of a machine one thousand times faster than current 
machines. There is no comparable technical progran in the U.S.. The Panel 
notes that in the case of the NASA Numerical Aerodynamic Simulator, a very 
high performance supercomputer, no acceptable proposals for its development 


12 

See, in Appendix III,. The Supercomputer Famine in American 
Universities , by L. L. Snarr. 

Supercomputer cycles are available at the University of Minnesota 
^ CRAY-1 and at Colorado State University CYBER 205. 

See, in Appendix III, the paper by R. G. Gillespie. Most supercomputer 
.c facilities are funded directly by the Federal Government. 

5 See, in Appendix I, Partial Inventory and Announced Orders of Class VI 
., Machines . 

See, in Appendix III, Japan's Initiatives in Large Scale Computing , by 
L. Lee. 
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thev woSld mlU ?Si ther / f - the two com P etin 8 vendors could assure NASA that 
would meet the performance requirements. Rather than developing new 

products, the vendors attempted to fit all NASA requirements to their 
existing product line. ' ' 

current^ ° f ? re !^ US studies » the Panel also finds that the power of 
current and projected supercomputers is insufficient to meet existing needs 

en ? lneer ;"e’ a " d technology, both military a*! civlua^ 7 
Research at universities and in the computer industry has indicated that 

of very h£gh PerformSnce computersm™? mty Sve 

^ohitlcturos 1 of°M eS . radl0ally different fr ™ the conceptually sequential 
architectures of today’s supercomputers. There are many candidate 

established^ 156 e y aluabed before conmercial feasibility can be 
established. . Simultaneously, the rapid and continuing advance of 

macHnpf ?^ 1C techn ? lo Sy makes ^ feasible to build such parallel 
machines. There is also a need for improvement of component performance. 

tha/thp 1 !!^ 1167 ?? ^ ha ^ unde [ curr ent conditions there is little likelihood 
at the U.S. will lead.in the development and application of this new 

advanced°d e velonmpnt eS * ,, Fa ? tors inhibitin g th e necessary research and 

advanced development are the length and expense of the development cvcle 

Very^igh SLfp ChlteC r re ’ ^ the uncertaint y of the racket place, 
very high performance computing is a case where maximizing short-term 

nation,?" Capital d ° eS not reflect the " a ti°nal security^r the long-term 
3 economic interest. The Japanese thrust in this area, though ™s 
public funding, acknowledges this reality. 

The Panel estimates 20 that the present annual investment in basic research 

dollars^whTip f° ftware > and architecture is between 5 and 10 million - 
dollars, while the annual advanced development expenditures for 

supercomputers (beyond Class 6 machines) are between 20 and 40 million 
dollars; This is contrasted with the development cost for a new highspeed 
convention 31 architecture system of approximately 150 million dollars as 

project i^JapSr^^Danpl 11110 "-^ 011 ^ 3 national superspeed computer 

project in Japan. The panel considers current levels of United StatP<? 
investments insufficient to maintain leadership in sSper-coSpuSrs 

The Panel believes that U.S. leadership in supercomputing is crucial for 
nationir^ul-Uyf “ lenCe te0h " ol °e>'. »»d therefore, for economic and 


17 


18 


19 

20 


See, for some illustrative examples, Appendix II. Also notable is 
Cha^an, 3 ?^] 301 " 2 ^ Ccfnputational Aerodynamics”, by D.R. 

See ’ m Appendix III, a series of contributions to the Wbrkshop In 
particuiar, the papers by Dennis, Gajski, et al., Ris, and Fe^nbach; 
~f° ^ he A rep ^^ °f Group 3 of the Lax Panel in the Supplenent. 

^’ ic Appendix III, Why the U.S, Government Shou ld Support Research 
on and Development of Supercomputers , by B. Buzbpp.-°--- 

Members of the Panel and of the Organizing Conmittee have conducted a 
survey to estimate the current total national investment in research 
and advanced development for supercomputer and supercomputing both 
public and private. This survey included research and develocment 
costs but excluded funding for the acquisition, maintenance, and 
operation of supercomputer facilities. 
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Recoriinendations , . 

The Panel recommends that the present needs and challenges to U. S. 
leadership in scientific computing and supercomputer technology be 
addressed with high priority. To this end, the Panel has set forth the 
background for planning and policy issues, outlined sane options, and noted 
that current total finding in this area is insufficient to maintain the 
Nation's leadership in large scale computing. The Panel has avoided 
recommendations of a prograrranatic and organizational nature; these, and 
their implementation, are best left to the appropriate government agencies. 
These agencies must work together to respond to the issues raised and pu 
together a detailed coherent program whose components are responsive to 
their individual missions. The program plan should contain a clear 
statement of goals, directions, and roles for tte academic, industrial, an 
Federal government segments; responsibilities of the participating Federal 

agencies; and funding required. 


The Panel recommends that a long-term National Program on Large Scale 
Computing should be initiated inmediately, with the participation of the 
appropriate Federal agencies, the universities, and industry. The goals of 
this National Program should be: 


1. Increased access for the scientific and engineering research 
community through high bandwidth networks to adequate and 
regularly updated supercomputing facilities and experimental 
computers; 

2. Increased research in computational mathematics, software, and 
algorithms necessary to the effective and efficient use of 
supercomputer systems; 

3. Training of personnel in scientific and engineering computing; 
and 

4. Research and development basic to the design and implementation 
of new supercomputer systems of substantially increased 
capability and capacity beyond that likely to arise from 
cormnercial sources. 


This Program should be coordinated by an interagency policy committee 
consisting of representatives of the appropriate Federal agencies, 
including DOC, DOD, DOE, NASA, and NSF. A Large Scale Computing Advisory 
Panel with representatives from Government, the universities, and 
industry, should be established to assist in the planning, implementation, 

and operation of the Program. 

The Panel finds two points that require emphasis: 

As the few successful facilities amply demonstrate, solution of the 
problem of access to supercoraputing facilities, on a national basis, is 

possible. 
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Secondly, the domestic computer industry must allocate its scarce 
research and development funds to meet all the commercial opportunities 
and competitive challenges. Supercomputing enjoys a priority within the 
computer industry. But this priority, which reflects competitive 
commercial conditions, does not reflect the entire national scientific 
and security interest. It is not reasonable to rely solely on industry's 
own initiatives and resources in this area. 

Possible Approaches for the National Program 

Ihe Panel has received many suggestions 21 for carrying out the thrusts of 
he proposed National Program. We outline here those considered most 
premising. 


1 • h£S. ev ]^ : There appear to be three approaches to provide reliable 

and efficient access to supercomputers to the research and 
development community. Common to all these is the development of 
a nation-wide interdisciplinary network^ through which users will 
have access to facilities. This netwo rk *hnniH ? ~h 

supercompu t er faciliti e s (ftXCftnt . thnsa ripd-inqf.pd i-n u P ry s ppoial 

tasks), including .commprrial—<;iip^p™nrv.4--i+ and - 
experimental machines . J * * - 

o The most expedient and perhaps least expensive way to provide 
supercomputer access to the broad range of scientific and 
engineering researchers is to enhance supercomputer capacity 
and staff at existing centers which have demonstrated 
sophisticated capabilities for providing large scale computing. 

o Provide supercomputers to selected government laboratories 
without such facilities and make them available to the broad 
research and development community through networking. In 
addition, there should be sharing and enhancement of current 
supercomputer facilities located at universities and government 
laboratories. 

o Establish additional regional centers at selected universities 
interconnected with existing facilities at other universities ’ 
and government laboratories. 


The existence of a national network would permit combinations of 
these nonexclusive options, as well as the appropriate use of 
commercial services. The mechanisms for funding these facilities 
and allocating access should be carefully studied/ 3 


21 


22 see position papers in Appendix III. 

' " The NMFECC network, described in Appendix I, is the exanple related!v 
mentioned for emulation because of its high bandwidth. The ARPANET 

23 " etw °[ k 1S of ten mentioned because of its interdisciplinary nature. 

See the position paper by R. G. Gillespie in Appendix III. 
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The above recommendations are related to the problems of access to 
general purpose supercomputer facilities. It should be noted, 
however, that there are scientific and engineering problems that 
can perhaps be better and more economically attacked by 
specialized supercomputing facilities and by sophisticated 
array processors. . The Panel recommends that, as part of the 
National Program, significant emphasis be placed on providing this 
specialized equipment to the research community. Finding the 
proper balance between investments on these two types of 
facilities requires a careful analysis, at the multidisciplinary 
and interagency level. 

2. Research in Software and Algorithms 

Today's supercomputers are a major departure from traditional 
sequential machines. Future significant improvements may have to 
come from architectures embodying parallel processing elements - 
perhaps several thousands of processors. In order to exploit 
today's vector processors and future parallel processors, entirely 
new algorithms must be conceived. Research in languages, 
algorithms, and numerical analysis will be crucial in learning to 
exploit these new architectures fully. The contributions of 
numerical analysis, computational mathematics, and algorithm 
design to the practice of large scale computing is as important as 
the development of a new generation machines. 

3. - Training 

Another important component of this National Program is the 
development of an imaginative and skilled user community in 
supercomputing. There is a considerable shortage of appropriately 
trained personnel and of training opportunities in this area. 

Forms of institutional encouragement, such as NASA's special 
fellowships in the area of numerical fluid mechanics, special 
summer schools, and special allocation of access time to 
supercomputers for those projects that involve graduate students, 
should be considered. Some of the more mathematical aspects of 
these activities .can be accomplished independently of the machines 


211 Some of these problems arise in a nunber of areas associated with 
experimental physics. See, in Appendix III, the letter from A. E. 
Brenner, Jr. See, also in the same Appendix, the position paper by K. 
Wilson on the use of array processors. 
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on which the actual calculations are done; however, the true 
integration of methods and their implementation cannot be done 
without access to supercomputers. The nature of the machine 
architecture has a very profound effect on the numerical methods, 
on the algoritlms, and of course on the software. Thus, while 
being trained, students must have.access to state-of-the-art 
computers. Today suoh training is virtually nonexistent; yet the 
skills gained from such training are essential to science and 
engineering. 

4. Research and Development for New Supercomputers 

There are serious component and architectural problems that must 
be solved as part of the development of future generations of 
supercomputers. The unique strengths of industry, universities, 
and Government laboratories should be brought together for this 
purpose. A group of panelists from this workshop, with the aid of 
a number of experts from industry and universities, has produced a 
report which describes one such program. 

Since a great deal of careful analysis and detailed planning is required 
before the proposed National Program can be implemented, the Panel urges 
that its recommendations be acted upon as soon as possible. 


25 _ 

See, in the Supplement, A Program for Development of Very High 
Performing Computer Systems , by J. C. Browne and J. T. Schwartz. 
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Wilson 


ON SUPERCOMPUTING 

K. Wilson 

Cornell University 
Summary and Recommendations 

The lack of large scale scientific computing* resources for basic 
university research has become a major problem for U.S. science. 

The largest need for these resources is in the theoretical science 
community, where they are required for computer simulation of complex 
physical processes that cannot te studied by traditional analytic means. 

The immediate national needs are the following: 

1) A national network linking all scientists involved in open basic 
research, vastly generalizing the existing. Arpanet and Plasma 
fusion energy networks. 

-2) A development program in support of large scale scientific com¬ 
puting, encompassing hardware, systems software, and algorithm 
development and carried out as a collaboration between knowledge¬ 
able members of the scientific community, the computer science and 
electrical engineering community, and the computing industry. 

5) Building an adequate equipment base (computers, peripherals, and 
network access) for training and theoretical research in 
universities. 


"large scale scientific computing" includes any problem requiring hours 
or more on a superminicomputer and any problem requiring major software 
development. It excludes routine evaluation of integrals or ordinary 
differential equations, and any problem solved on a pocket calculator. 
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4) Providing adequate access by researchers on the network to special 
and general purpose facilities at-the National Laboratories and 
elsewhere, for computing needs that go beyond the base level. 

An interagency committee should be established to channel 
support for scientific computing into relevant agency programs funding 
basic research. The interagency committee requires a strong scientific 
advisory board, thoroughly knowledgeable about both scientific computing 
and developments in computing technology, to set priorities and make 
sure that long term as well as short term needs are met. 


Background 

A major change has taken place in the role of computers in basic 
scientific research. The use of large scale scientific computing for 
simulation is growing very rapidly in universities in very many areas 
of basic theoretical science. Previously, the principal uses of computers 
have been for data collection and analysis (such as in high energy 

experimental physics and space physics) and in very specialized areas of 
theory (for instance in specific areas of fluid dynamics or in plasma 

fusion studies), where there are major simulation efforts centered at 
the national laboratories. 

Many of the simulations that have been performed recently in 
universities are of a preliminary nature. Numerical approximations 
used have been very rough; important aspects of the science involved 
have been simplified or neglected altogether. The preliminary results 
have been promising, but no more; the preliminary work must be followed 
up by much more careful computations taking into account all important 
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scientific principles and including careful determination of numerical 
truncation errors. In a number of areas, further progress in both 
analytic and numerical studies is dependent on reliable completion of 
simulations that are in this preliminary stage. Specific examples of 
such simulations in elementary particle theory, statistical mechanics, 
and astrophysics are given in Appendix 

A major instrumentation program, outlined above, is required to 
enable completion of the existing simulation efforts. The nature and 
magnitude of computing capacity that will be needed varies considerably 
from case to case. A number of computations in the very general area 
of fluid dynamics could be completed if computing access in universities 
was made comparable to that presently available at national laboratories. 
Other computations, in elementaiy particle theory and statistical 
mechanics or turbulence will require computers to be built that are very 
much more powerful than any current supercomputers. See Appendix 
for examples of computing support needs. 

Several other changes have taken place that underscore a need for 
a computing initiative for theory. 

Sciende is becoming more interdisciplinary than ever before and 
faces new demands from the high technology economy. To meet these needs 
a new level of interchange must take place between scientists in widely 
different disciplines. For example, renormalization group ideas are 
being pursued in subjects ranging from quantum field theory to oil 
propagation in reservoirs to polymer folding problems; scientists in all 
these fields must have access to the latest developments in all the 
other areas. Computer networks provide extraordinarily powerful means 
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of communication that cannot be duplicated otherwise. A researcher on 
Arpanet can broadcast a request for information about a subject totally 
outside the re searcher's own field nf-i»Ypprt^ P anH p ^ pprt ypplies from 
e xperts on the subject asked about . These communication capabilities 
will be greatly enhanced by use of the emerging technology of "desk 
top work stations". The newest generation of scientists, who see this 
technology coming and understand its extraordinary capabilities for 
improving scientific productivity and making science more rewarding, are 
becoming demoralized because of continued delays or inaction in making 
network access generally available nationally or even within single 
universities. 

Another recent development is the soaring enrollment in computer 
science courses nationwide as students realize that they must be 
prepared to work with computers regardless of their field of study or 
later career. Estimates at Cornell are that 75% of all undergraduates 
will shortly be taking the basic computer science course. The time has 
com e to provide training in large scale scientific computing throughout 
undergradua te and grad uate programs for all science majors . A consider¬ 
able upgrade of current facilities will be required to support the training 
effort. There can be no flinching from introducing students to verv 
• powerful computi ng, since by the time thry g r aduate and have 
work force, the most powerful computers available while they were 
students will have become cheap and commonplace, and graduates will have 
be a k lc -,t° with them, effectively to be competitive in the 


work force. 






The large scale computing scene and the role of universities in 
this scene is changing. A great variety of limited purpose computing 
systems are emerging from the computing industry which will be as 
important as today's mainframes and supercomputers. Even a listing of 
current or shortly to be delivered systems illustrates this diversity 
the Floating Point Systems attached processors, the ICL DAP and the 
Goodyear MPP, the Denelcor HEP, and the CDC Advanced Flexible Processor 
all have unique architectures and each has strengths for specific 


application areas that mainframes or vector supercomputers cannot match. 

Each new large scale computing system with a new architecture 
and new software faces a long breaking in period before the hardware 
and software work with sufficient reliability to support large user 
communities. Universities provide the most effective fra mework for th e 
breaking in of a new computer. University faculty carrying out large scale 
simulations need access to the most cost-effective computing systems coming 
on the market in order to complete their work; graduate students and 
undergraduate work study students help overcome the innumerable minor problems 


new systems present; computing support personnel use this experience to 
help the computer manufacturer find and correct problems early in the 
product cycle. Computer science students can work with computing support 
personnel to flesh out inadequate software; new architectures can provide 
inspiration for major projects in computer science and electrical engin¬ 
eering. A university installation showcases a new computing system for the 
entire world. The Array Processor project at Cornell has already illustrated 
most of these capabilities of universities. 
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Despite the enormous benefit an early university installation provides 
to a computing manufacturer, the manufacturer rarely can afford to provide 
the system for free. However, there are many ways a manufacturer can .reduce 
the costs to a university for a new system, and this in turn can provide 
major cost reductions for a governmental computing initiative. At the 
same time that the government saves money, the U.S. computing industry can 
be strengthened against major international competition by the help 
universities can provide in bringing new systems to market. 

Because of the great variety of new computing systems that need to be 
placed in universities in support of the simulation effort a national 
computer network is e ssential to enable individual researchers to work 
remotely on the computer mosTl^Itc 'd to their problems, and to equalilP 
ac cess among diffe rent university an d national laboratory researchers to 
the computing systems that exist. 

A strong basic resear ch effort involving computer simula tion is 
essential to the health a£-i n du ^.ri a 1 tps.p.-^h ^v r ln r nr nt industrial 
products and processes arc developing rapidly in response to increasing 
international competition, changing raw material mixes, etc. Comput er 
simulation, when feasible, is usually the least expensive and most rapid ' 
method available for prototyping and nntimi ration nf ^ p roducts 
processes. Use of computer simulation must grow rapidly in industrial 


research and development if industry is to stay competitive. Unfortunately* 
the bulk of U.S. scientists today arc both ignorant and highly skeptical 
of large scale computer simulation, and they bring this attitude to industry, 
whether as employees, consultants, or members of industrial advisory boards. 
It takes many years of practical experience with large scale computing to 
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be able to make sensible decisions or provide sound advice on simulation 
matters; very few U.S. scientists have this experience. The leaders of 
advanced research groups in industry who must build up aggressive computer 
simulation programs are often very isolated; they typically face higher 
level managers with little computing experience, computing support personnel 
with more concern for business data processing than scientific computing, 
and scientific support staff with little experience especially in large scale 


software. 

The current simulation effort in universities sometimes involves 
subjects (such as turbulence) of major industrial engineering importance. 
Even in areas of no practical significance, such as general relativity, 
numerical simulation experience is invaluable training for students movin 
to industry and helps build a reservoir of university faculty with compe¬ 
tence in large scale computing matters. 

A number of the university simulation efforts if allowed to continue 
will encounter the same software complexity barriers that plague industry 



and governmental laboratories. Industry cannot do much about the software 
complexity problem in simulations because there is too much pressure from 
the backlog of production software to maintain and incessant deadlines to 
meet. The computing industry itself cannot do much because they have 
little expertise in the area of software for scientific'simulation and the 
scientific simulation market is not large enough for its software problems 
to be given high priority. 

The computing support initiative should include very strong encourage¬ 
ment for university scientists engaged in simulation to encounter soft- i 

\ 

ware complexity problems and then collaborate with computer scientists to 


Wilson 


find long term solutions to this problem. Extraordinary ideas are 
circulating within the computer science community that can help deal with 
software complexity issues, but scientists are reeded to help package .the 
computer scientist's ideas in working software tools that will provide 
maximum benefit in the scientists' work. The computing resources presently 
available to university theorists are not powerful enough to allow major 
software difficulties to develop. Some of the major data processing efforts 
in experimental science have encountered software problems, as well as 
major productivity losses due to software development hassles, but the 
problems have never been severe enough to motivate the experimentalists to 
address the software complexity problem directly. There is already an 
awareness among a few members of the theory community that software 
problems are imminent and must be dealt with. 

Justification for the Recommendations 

1) The need for a national network linking all scientists has already 
been justified. The network must support computer mail, bulletin 
boards, newsletters, etc. which provide the basis for interdisci¬ 
plinary communication. The network must support remote interactive 
computing sessions and computer-to-computer file transfer. Many 
researchers already have informal access to Arpanet, which provides 
these facilities but many others do not. 

2) A major development program is needed in conjunction with any 
computing support initiative. The continued rapid development of 

VLSI (very large scale integrated circuits) is opening up a bewildering 
variety of computer architectures and designs, mostly of a special 

purpose nature. High performance computer design (beyond 
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mainframe levels) will necessarily become a collaborative effort 
between a computing manufacturer and the scientists who will use the 
computer or their representatives, illustrated for example by the, 
collaboration between NASA and Goodyear Aerospace Corporation on 
the design of the MPP image processing system. The architectures 
of these systems pose major software problems that the computing 
industry is ill-equipped to handle, especially for special purpose 
systems with limited markets. Scientists will have to seek help 
from computer scientists to get advanced software systems developed for 
new special purpose hardware, as NASA has already done for the MPP; this will 
provide a major source for computer science thesis projects. Even the most power 
computers modern technology can produce will not be sufficient for 
some of the most demanding simulations unless major advances are 
achieved in the algorithms used to represent problems numerically; 
scientists must collaborate with numerical analysts and applied 
mathematicians to seek the most efficient and reliable numerical 
techniques for their simulations. 

The base level of computing support in universities must be con¬ 
tinually modernized and built up in order to provide students with 
training in large scale computing that is not hopelessly obsolete 
before they graduate and to provide an informal atmosphere for 
exploratory computations. In the near future an up-to-date uni¬ 
versity computing support system would begin with a desk-top work 
station on every faculty member's desk and other work stations 
easily accessible to every student (graduate or undergraduate). 

These work stations would be linked via high speed networks to form 
clusters around a central "server” with disk storage space, printers. 
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and modest graphics facilities. The "server" would itself be linked 
to other campus facilities (including centralized very high performance 
graphics) and the national network. Inexpensive, high performance 
computers (such as attached processors) should be provided locally 
in sufficient quantity to minimize the remote computing load on the 
national network, while supporting heavy student and faculty use. 

4) The National Laboratories have a long record of providing high perfoimance 
computing. Their non-secret facilities need to be strengthened and made 
accessible to university researchers on a national network. A variety 
of high performance supercomputers and more specialized systems need 
to be provided to universities and likewise made available on the 
network. University sites for these systems should have effective 
centralized management and adequate interest and support to bring a 
new system into full operation quickly and economically. The most 
important function of these facilities, once in full operation, 
should be to enable completion of the scientifically most urgent 
simulations, in preference to work of lesser significance. Finishing 
a simulation, with reliable error analysis, can require orders of 
magnitude, more computing power than the initial exploratory compu¬ 
tations; careful and unbalanced resource allocation procedures will 
be needed to provide maximum scientific return from supercomputers 
and other expensive systems. It is not possible on any reasonable budget 
to allow all scientists unlimited access to these systems and still 
avoid total saturation and long job turn-around times which reduce 
user's productivity; severe allocation procedures will be required to 
prevent this saturation. 
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NEEDS AND OPPORTUNITIES FOR SUPERCOMPUTERS 


H.B. Keller and J.R. Rice 


J_. Introduction . 

It has been widely observed that a new discipline 
Large Scale Scientific Computing - has been born. Many make 
the case that science and engineering is no longer divided 
into just two parts - theoretical and experimental; there is 
now a third equal part - computational. There is no 
national policy recognition of this new and basic scientific 
and engineering discipline nor of the importance that super¬ 
computers, its main tool, will play in the future develop¬ 
ment of technology. The needs and opportunities that we 
shall address are concerned with long range and basic 
research goals. There are any number of short range, prac¬ 
tical and very important accomplishments that can and will 
be made. But we are here concerned with the future of one 
of the basic developments in twentieth century science and 
technology and we are determined that the United States 
should continue to lead the world in these developments. 

Supercomputers are a new kind of instrument for science 
and engineering. Telescopes and particle accelerators come 
to mind as analogies and, indeed, such instruments will 
always continue to have a basic influence on scientific 
developments. Such devices enhance the ability to observe 
the consequences of basic laws of physics on the largest and 
smallest scales possible. However, supercomputers enhance 
man's ability to reason rather than to observe. This is a 
completely different type of activity - it cuts across all 
fields - and we have not yet begun to see its ultimate 
implications. The possibilities are so profound that we 
must ensure that we are at the forefront in these develop¬ 
ments . 

The next section describes eight areas where supercom¬ 
puters will have major impacts that are critically important 
to the economic and military health of the country. There 
are many more opportunities in the areas (see Section III) 
which are now or soon will be involved in large scale 
scientific computing. These areas represent the specific 
needs and opportunities for large scale scientific comput¬ 
ing. The general needs and opportunities have not been 
properly recognized before. This is surprising because they 
are common to all users of large scale scientific computing; 
in the past the support has been justified in each specific 
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term weather patterns. 

Nuclear weapons . The analysis and design of 
weapons currently consumes enormous amounts of 
power and yet drastic simplifications are made 
situations. 

Electronic devices . Supercomputers are now being used 
to design both the circuit layouts and the individual 
components of electronic chips. Yet current designs 
are all two-dimensional and it is clear that much 
greater performance (i.e. much cheaper chips) can be 
obtained with three dimensional designs and manufactur¬ 
ing. The computational requirements are so formidable 
that such chips are only in science fiction - for the 
m onent. 

Command and control . Chemical refineries and power 
plants are now controlled by rather modest computers. 
Automated assembly lines with many robots require total 
computing power equal to that of the current biggest 
supercomputers. The analysis and response to an 
intense attack will require a Navy ship to have a 
supercomputer more powerful than any that exists now. 
Computational requirements increase dramatically as the 
complexity of the situation and response- speed 
increases. We are just beginning to imagine what 
supercomputers can do in this area. 

Disease control . Even simple viruses are enormously 
complex molecules, it is a major computational project 
just to determine their structure. In principle, we 
can use simulation to determine their chemical 
behavior, test their reaction to various drugs and 
finally understand how to control them. This (and many 
other medical advances) must await computers that are 
hundreds of times more powerful than anything that 
exists now. 


We have omitted many equally important areas (e.g. nuclear 
power plant accidents, circulation of the ocean, magnetic 
fusion energy, satellite photo analysis), but the nesaage is 
the same: a country that wants to be at the forefront scien¬ 
tifically, militarily and economically, must have access to 
the best computers, the supercomputers. 

3. DISCIPLINES AND PROJECTS USING LARGE SCALE SCIENTIFIC 
COMPUTATION 

Several disciplines and projects have already demon¬ 
strated the value of Large Scale Scientific Computation. 
Common to all of these areas is the need to solve extremely 
complex problems; so complex that, to date, only gross 


nuclear 
computer 
in most 
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simplifications of the full problems have been attacked with 
the aid of supercomputers. Included in this group of dis¬ 
ciplines and projects which already have dedicated supercom¬ 
puters are: 


A1 . 

Nuclear weapons research 


A2. 

Atmospheric sciences 


A3. . 

Magnetic fusion energy research 

A4. 

Aeronautical research and 

development 

A5. 

Nuclear reactor theory and 

design 

A6. 

Petroleum engineering 


A7. 

Geophysics 


A8. 

Intelligence (classified) 



There are numerous other disciplines and projects that 

° f thC 3ame ° r greater computing power and that 
1 °* yet * ave xt in an y organized way. These areas are 
-or the most part actively engaged in Large Scale Scientific 
Computation and have begun to try to acquire supercomputers. 
They are grouped as follows: * 


B1 . 


Computational physics 


B2. Computational mechanics and structural design 
B3* Ocean sciences and underwater acoustics 

B4. Computational chemistry and chemical engineering 
B5. VLSI and circuit design 

B6. Nonlinear optics and electromagnetic theory 
B7. Computational fluid dynamics 


In addition there are many areas that will shortly realize 
that Large Scale Scientific Computation is either vital to 
their continued development or can play a large role in 
solving some of their basic problems. These areas include- 
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Cl . 

Astrophysics, 

planetary 

science and astronomy 

C2. 

Economic modelling and o 

pe-rations research 

C3 • 

Biosciences 



C4. 

Computational 

statistics 

and graphics 


The above listed disciplines and projects are fre¬ 
quently interrelated, they are not merely of academic 
interest and include work in government laboratories and 
industry. Thus, the interest in and applications of Large 
Scale Scientific Computation clearly cut across all the 
standard scientific, technological, industrial and govern¬ 
mental lines. 


A brief bibliography of documents addressing the need 
and opportunities in the areas covered above are contained 
in Supplement. In essentially all of these discussions the 
case is made that new knowledge and understanding of 
phenomena ranging from the basic laws of physics to the 
behavior of a nuclear power plant during an accident can be 
attained if more powerful computing equipment is available. 
Each significant increase in computing power can lead to a 
host of significant advances in each discipline or project. 
Thus, the effect of improved computing power can be multi¬ 
plicative if the opportunity is taken of making this power 
broadly available. 


4. 


LARGE SCALE SCIENTIFIC COMPUTING: 


A New Discipline 


The basic use of computing in almost all of the above 
areas is either: 

(i) to approximate the solution of complicated systems of 
nonlinear partial differential equations, 

(ii) to model or simulate complicated physical phenomena or 
systems in terms of interacting simple systems. 


These two procedures are identical in many cases. As the 
phenomena becomes more complex or the models more realistic, 
the demands on computing power increase rapidly. For exam¬ 
ple, in typical equilibrium fluid dynamics problems '(e.g. 
simulation of a plane in Bteady flight) going from one to 
two or from two to three space dimensions increases the com¬ 
plexity (as measured by the number of unknowns) by a factor 
of 20 to 200. The operational count to solve equilibrium 
fluids problems is proportional to the cube of the number of 
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•unknowns and thus it increases by a factor of from 8 x.10^ 
to 8 x 10^. It is thus abundantly clear that the need for 
continual improvements in computing power will be with us 
for some time to come. * In all fields using Large Scale 
Scientific Computation, the indicated estimate or very simi¬ 
lar ones are given to show that increases of the order of 
. a 6 

10 or greater in computing power are needed and will be 
extremely beneficial. 


The payoff in being able to solve two or three space 
dimensional problems (compared to the previous one or two 
dimensional cases) can be enormous. Drag reduction calcula¬ 
tions over full three dimensional ships or aircraft could 
alone save billions of dollars in fuel costs. Accurate 
simulation of three dimensional two-phase flows in oil 
fields could increase secondary and tertiary recovery by 
well over 10? - 20$. This is equivalent to finding oil 
worth over $100 billion. 

The entire Large Scale Scientific Computation community 
agrees that future supercomputers will operate in a parallel 
manner. They also agree that it is extremely difficult to 
plan software, algorithms and numerical methods that will 
take full advantage of this parallelism. Furthermore people 
will just not devote themselves to such difficult tasks 
unless they can have access to the supercomputers to test 
their results. 

The role of software in current supercomputers already 
shows the striking but disturbing features to be expected. 
On one of th<j most basic algorithms (Gaussian elimination) 
which pervades Large Scale Scientific Computation , the use 
of different Fortran compilers and some "tweaking" with 
handwritten code produced running programs with a maximum to 
minim urn speed ratio of 60. All the compilers used are con¬ 
sidered quite good! Such gross variations do not occur on 
the standard software for serial machines. Thus, even with 
the currently existing minimal parallelism (or rather pipe¬ 
lining) we do not yet know how to produce near optimal 
software. It will be even more difficult for the coming 
generations of supercomputers. 

The contributions of numerical analysis and algorithms 
design to the practice of Large Scale Scientific Computation 
is, in a very real sense, as important as the development of 
new generation machines. That is the speed up or 
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improvement on what can be effectively computed comes as 
much from the numerical methods and their algorithmic imple¬ 
mentation as it does from hardware improvements. Indeed a 
recent study (j.R. Rice, Numerical Methods , Software and 
Analysis , McGraw-Hill, 19821 of aTgorithms for solving 
elliptic problems in three dimensions found a speed up of 

5 x 10 10 from 1945 to 1975! This is much greater than the 

improvement in going from in I.B.M. 650 to a Cray-I. These 

estimates do not include mulit-grid methods, currently in 

active development, and it is reasonable to assume another 

3 

factor of 10 will have been achieved during 1975-85* It is 

clear that Large Scale Scientific Computation is the most 

significant and powerful scientific instrument devel.oped in 
the 20th century, the development is not nearly complete and 
that very shortly it will not be possible to do first rate 
scientific research in many areas without the best supercom¬ 
puters and methodology of Large Scale Scientific Computing. 

It seems generally to be the case that discipline or 
project oriented scientists do not use the latest or best 
numerical methods or algorithms. Also, it is rare, but not 
unknown, that improvements in numerical methods or algo¬ 
rithms are made by such project scientists using Large Scale 
Scientific Computation. Thus, an important aspect of our 
program must be an attempt to close this gap between 
development and use of new ideas. 

5 . GENERAL NEEDS 

The general needs in large scale scientific computing 
are in four categories: 


1. Trained people 

2. Software systems 

3 • Algorithm design and numerical analysis 

4. Supercomputer hardware and systems 


Some view that the last category as part of electrical 
engineering and/or computer science hardware. Where it is 
included is not nearly so important as that it be recognized 
and adequately supported. 
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A PROGRAM FOR DEVELOPMENT OF VERY HIGH PERFORMANCE 

COMPUTER SYSTEMS 


1.0 EXECUTIVE SUMMARY 

The premise upon which this report is based is that 
continued leadership in very high performance computing and 
its applications are crucial to the United States both for 
national security and for economic development. The rapid 
and continuing advance of microelectronic technology is 
opening a path to the development of a new generation of 
supercomputers which can potentially obtain computation 
rates two or three orders of magnitude faster than today's 
most powerful machines. Development and application of 
these very high performance computers is crucial for both 
national security and long-term economic development. 

Research has established that future generations of 

very high performance computer systems will have parallel 

architectures radically different from the conceptually 

sequential architectures of today's supercomputers. There 

are many candidate architectural proposals which must be 

evaluated before commercial feasibility can be established. 

There is, at present, little likelihood that this research 

will be developed and exploited in this country given the 

length and expense of the development cycle for a radical 

new computer architecture and the numeric size of the 

marketplace. Immediate action on these opportunities is 
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essential if U.S. leadership in very high performance 
computing is to continue. 

The program presented herein has as its first goal 
acceleration of the development of current research concepts 
for very high performance computer architectures into 
commercial products. The goal can .be accomplished by a 
research and development program which brings very high 
performance computer technology to a state where the normal 
venture capital mechanism will select it for commercial 
development. 

This program is required because the unfavorable 
short-term risk-to-return ratio for development of a 
radically architectured class of supercomputers totally 
inhibits operation of the normal venture capital mechanism. 
Very high performance computing is a case where maximizing 
short-term return on capital does not reflect the national 
security and long term national economic interests. 

The program is founded on a development model based on 
the concept of university/industry/government laboratory 
collaboration. This development model is appropriate to the 
U.S. economic system and culture and may be a procedural 
prototype for acceleration of application of research in 
other high technology areas with unfavorable short-term 
risk/return ratios, but where long-term national significance 
development is concerned. 
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The very High Speed Integrated Circuit (VHSIC, program, 
sponsored by the Department of Defense, is developing the 
component technology base for high performance parallei 

computing. The application for which the VHSIC technology 
is intended will require very high performance parallel 

architectures in order to meet their objectives. This 
Program focuses on accelerated development of the 
architectures and systems which will utilize the VHSIC 
device technology and is thus a natural successor of the 

VHSIC program. The absence of such a program may lead to 

the Japanese beina _ ■ 

9 principal beneficiaries from 

commercial application f «.u 

of the enhancement of component 

technology driven by the VHSIC program. It should be noted 

VHSIC program does not address several component 
technology issues crucial to develo^ent of general purpose 
very high performance computer systems. 


2.0 PROGRAM JUSTIFICATION 


The current 


applications of super 


computers 


include 


of 


design and simulation 
VLSI chips 
of nuclear weapons 
and analysis of 
nuclear reactors 
energy research 
directed en€>rgy weapons 
intelligence applications 
aerodynamics, structural 

design and evaluate 
structural mechanics 


design 

design 

fusion 


electric power distribut 
atmospheric science 
oceanography 
fluid dynamics 
automobile design - 
design of manufacturing 
systems 
geophysics 

petroleum exploration 
and reservoir 
management 


Potential 


but not yet 


exploitable applications 
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time image processing and fast robotics. The list of 
potential applications does not include artificial 
intelligence oriented applications such as expert and/or 
knowledge based systems. These problem domains may require 
very high performance parallel architectures for effective 
application. The software algorithms and user interfaces 
are so different from the essentially numerical applications 
as to justify separate consideration. (See Appendices B and 
H for additional information.) 


Electronics, nuclear weapons, intelligence, 
aerodynamics and energy production are currently major 
components of U.S. defense. Progress in each of these 
areas will be paced, in fact may be bounded, by progress in 
high performance computing. The same is true for most of 
the other applications and potential applications. Thus, 
leadership in supercomputers is fundamental to U.S. defense 
and to U.S. leadership in crucial areas of technology. If 
another country should assume leadership in supercomputers, 
U.S. defense and technology will depend upon access to 
computers of foreign manufacture. This presents three 
risks. 


1. Currency of_ Access . If other countries consume the 

first two years of production then U.S. scientists 

and engineers will be denied use of these machines 

in U.S. technology and weapon system design and 

defense application could lag theirs by that 

(Hitachi has recently announced a very 
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high performance array processor which will not be 
marketed in this country.) 

2 * Denial of Acces s. in this worst case U.S. 
technology development could be handicapped until 
domestic sources are developed. 

3. Computing Technology Lag . Development of 

supercomputers has always driven the development of 
other computer systems and been an important driver 
of electronics technology development. if another 
country assumes leadership we may lose these 
benefits. 


These risks are sufficiently crucial to both national 
security and national economic development to be 
unacceptable. 

The United States currently leads in the development of 
supercomputers and will be expected to do so for the next 
two to three years. Continuing American leadership in this 
area is threatened by two factors. One factor is the 
unfavorable short-term risk/return ratios which attach to 
the major innovations required to the development of new 
supercomputers and block the normal venture capital path for 
the development of research into commercial products. The 

factors which yield the unfavorable risk/return ratio 
include 


- 47 - 



Browne & Schwartz 


1. There are many alternative courses for development 
of parallel architectures which must be evaluated. 

2. Parallel architecture will require an entirely new 
generation of software and peripherals. 

3. The long (5-8 year) development cycle 

4. The large capital requirement 

5. The existence in the computer system marketplace of 
many low risk, short term, high return 
opportunities 

6. The relatively small market. There are now 
approximately fifty current generation "super 
computers" now installed. 

Short-term risk/return assessment for product development 
does not reflect long term national economic and security 
interests. 

The second factor is that the Japanese government and 
computer industry, having recognized the crucial nature and 
economic significance of the development of supercomputers, 
are very strongly committed to a research and development 
program in this area. The Japanese are now engineering for 
commercial exploitation machines based upon the research 
done in this country in the 1970's. This last statement is 
particularly significant. It points to the crucial element 
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which must be supplied to continue U.S. leadership. That 
is, commercial exploitation of research concepts whose 
development cycle is too long and complex for operation of 
the venture capital avenue which has been so dramatically 
successful with short term pay-out concepts. The program 
proposed has as its goal lowering of market risk and pay-out 
cycle to where the normal venture capital mechanism will 
operate effectively. 


3.0 description of proposed program 


This section outlines a program designed to lower the 
market risk and development costs of a new generation of 
very high performance supercomputers, to a level where 
development and marketing by private computer vendors will 
become not only feasible but attractive. The program 

mobilizes the present strength of U.S. research groups, 
government laboratories and computer vendors. The program 
has three major focuses. The key concept for accelerating 
development is to combine the creativity of the university 
research programs, the engineering expertise of computer 
vendors and the practical concerns of users and potential 
users of very high performance computers. It is also 
important to note that there are now available limited 
capability systems which will support concept evaluation and 
application development for parallel systems. The functions 
of the program will be.to: 
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1. accelerate the development of current and future 
generations of parallel architectures to be the 
bases for new very high performance computers, 

2. establish a knowledge base for application of 
parallel computing concepts to problems of national 
security and economic interests and 

3. broaden and enlarge the base of research activities 
to ensure the continuation of the flow of 
innovative architectures in very high performance 
computing concepts while at the same time diverting 
a significant fraction of the research community in 
very high performance computing to development. 

The steps in the development and application of a major 
new computer architecture are shown in Figure 1. The 
proposed program has as its major thrust acceleration of the 
product development and application cycle represented by 
steps 2, 3 and 4 of Figure 1. It also specifies 

strengthening basic research in the area of very high 
performance computing. 
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Basic Research in (1) 
High Performance Computing 


--+ I 

I I 

V V 

scalable, feasibility 
demonstrations 


V V 

commercial prototyping 


I I 

V V 

System Application and 
Evaluation | 


Figure 1: Stages in Development of Very High Performance 

Computer Systems 
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There are four activities in the proposed program, 
supporting all of the development phases of Figure l. These 
activities are: 

a. Accelerated Development of S calable Fe asib ility 
Demonstrations . 

Proposals for exploratory development of 
significant research architectures will be received and 
considered for funding as accelerated development 
projects. it is anticipated that there will be formed 
university/industry/user consortia to- develop scalable 
feasibility demonstrations of the most promising of the 
architectural concepts established by current research. 
The scalable feasibility demonstration must include 
development of significant applications to demonstrate 
the capabilities of the system. 

• Commercial Prototyping . 

Projects whose scalable feasibility demonstrations 
establish significant commercial potential will become 
candidates for construction of a full scale commercial 
prototype. This will involve a supported development 
effort with focus shifting to industry and which is 
expected to lead to a product that industry will 
subsequently carry forward on a commercial basis. 

Evaluation and Application . 

There will be established application and evaluation 

laboratories based upon the existing products which 
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provide prototype products that illustrate the concepts 
oi: parallel computing. 

d. Basic Research . 

The funding for basic research in very high performance 
computing will be enhanced in order to continue the 
flow of new research architectures and concepts. A 
crucial component of basic research which requires 
immediate emphasis is development of algorithms for 
effective utilization of parallel architectures. 

This development program may serve as a model for 
shortening the research to product cycle in other high 
technology/high risk areas which have poor short-term 
risk/return ratios, but which have long term national 
interest significance. The panel believes that the 
recommended program will, within the context of the U.S. 
economic system, dramatically shorten the duration between 
research and product in the critical area of very high 
performance computer systems. 

All of the candidate projects for accelerated 
development exhibit some form of parallel architecture. The 
projects selected for accelerated development must include 
not only hardware architecture development but also research 
and development on al 9 orithms, software systems, languages 
and the peripheral and supporting facilities which are 
necessary to make supercomputers usable. 
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The accelerated development proposals may focus on 
general purpose systems with very high performance or on 
special purpose systems which have broad application across 
a spectrum of disciplines such as real time image 
processing . 


4.0 ORGANIZATIONAL AND ADMINISTRATIVE STRUCTURE 


The panel proposes an administrative structure 
consisting of three groups: one to set policy# one to 

coordinate and administer the program and a technical 
advisory body to work with the coordination and review 
groups. Figure 2 shows the relationship of the proposed 
groups one to another as well as their relationship to the 
federal agencies which will make ultimate funding decisions 
concerning projects to be supported. 


+- 

| Computational 
| Science Policy 
j Board 

+- 


+ 


I 

I 

+ 


I 

+ — ------+ 

| Interagency I — 

| Working Group I 

+ - + 

/I I \ 
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I 
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| Technical 
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I 

I 
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Figure 2 
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The Computational Science Policy Board (CSPB) will 

oversee program and policy issues. The board will review 

resource allocations among the agencies for consistency with 
the overall policy. 


The Inter-Agency Working Group (IWG) will allocate 
projects and budgetary resources among the agencies. The 
membership of the group will come from the participating 
agencies and will also include the Chair of the policy 
board. The IWG will coordinate budget allocations, 
requirements and plans among the agencies. 

The Technical Board (TB) will assist the CSPB and IWG 
in technical matters. It will not have direct budgetary 
responsibility. It wi11, however. be used to assist in 
evaluation of proposals and advise on distribution of agency 
resources for project execution. 


It is anticipated that 
development projects may be 
first year of the 
development project 
year. The 
initiated 
prototype 
more than 


U P to five accelerated 

the 
each 
per 
be 
Each 
No 
any 


active at any time after 
program. The average cost of 
will not exceed five million dollars 
first commercial prototype development will 
in year three or four of the program, 
construction will take three to five years, 
two prototypes should be under development at 


one time. 


The budget given 
million dollars to 


in Table l provides an additional ten 

fund evaluation of current and future 
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prototypes, to begin development of parallel formulations of 
applications of national security interest and for 
additional funding for research in VHP computing. 

Basic 



Research 
and System 
Evaluation 

Scalable 
Feasibility 
Demonstrations 

Commercial 

Prototypes 

Total 

1984 

10 

15 

- 

25 

1985 

12 

20 

- 

32 

1986 

14 

25 

5 

44 

1987 

15 

25 

15 

55 

1988 

15 

25 

30 

70 


Table 1: (figures are millions of dollars) 

The basis for selection on program activities a, c and 
d will be unsolicited proposals. It is anticipated that the 
accelerated development proposals will originate with the 
university/industry/user consortia and prototype 
construction proposals from vendors. Proposals of any 
structure will, however, be considered on merit. 
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APPENDIX A 

Statement of Problems 


The technological objective for long-range 
supercomputer development, we believe, should be the 
availability by 1990 of computers 1000 times more powerful 
than the current generation of supercomputers. There are 
serious problems that must be solved to reach this 
objective, but the perception of the research community is 
that solutions to the problems can be attained within this 
time frame. The difficulties of designing systems of such 
extraordinary power can be broken down into a matrix which 
is shown in Figure A-l. Going across the three principal 
parts of a computer system are listed. The "engine" is the 
CPU and memory; the "system" is the set of high speed 
peripherals (disks and tapes) directly attached to the 

engine, and the "satellites" consist of graphics, 
communications and slow—speed peripherals. 


Reading down, the basic issues of computer development 
and use are listed: software, components, architecture and 
algorithms. "Components" is understood to include all 
aspects of hardware such as chip design, packaging and 
cooling, while architecture refers to global aspects of the 
computer design, such as vector vs. parallel organization. 
We have listed the degree of difficulty faced for each entry 
of the matrix. "Critical need" means there are problems 
that must be solved to reach the objectives. "Marginally 
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acceptable" means that it should be possible to live with 
normal technological development but there are problems that 
ought to be attacked. "Satisfactory" means we anticipate no 
serious problems. 


The really crucial software issues of language, 
programming environments, etc. are subsumed under the entry 
for "engine". 


ENGINE 

Software critical need 

Components critical need, 

device aspects 
being considered 
by VHSIC, attention 
to packaging and 
cooling is required 

Architecture critical need 


Algorithms 


critical need 


SYSTEM SATELLITES 

critical need satisfactory 

critical need some deve 

for a majority needed 
of problems 



critical need 
for some 
applications 

critical need 
for some 
problems 


satisfactory 


critical nee< 
for some 
problems 


Figure A-l 
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APPENDIX B 

The Japanese Supercomputer Program 

The Japanese government and computer industry have 
noted the tremendous opportunities and leverage potentially 
available through development o£ such computers, and have 
established two major projects in this area: a 
scientifically oriented "supercomputer" project and a "fifth 
generation computer" project with a strong artificial 
intelligence orientation. These projects are coordinated 
and funded by the Japanese government industrial agency 
MITI, and involve all of the major Japanese computer 
companies. 

An extended visit to Japan was recently made by 
visitors from Los Alamos Scientific Laboratories and 
Lawrence Livermore National Laboratories. We quote a report 
of these trips to illustrate the commitment of the Japanese 
government and computer industry. "The supercomputer 
project calls for the development of supercomputers for 
scientific and technological applications... Funds cover 
software and hardware development... The supercomputer 
association (established to manage this project) has a 
contract with the Japanese government. Its six member 
companies are Hitachi, Fujitsu, Mitsubishi, NEC and qfci 
Electric... The purpose of this project is to 'develop an 
ultra high speed computing facility for scientific and 
technological applications that is more than one thousand 
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times faster than "high-end" conventional machines. 1 " 

MITI has also established a "fifth generation computer" 
project which is controlled by another association whose 
membership comprises the same six Japanese computer 
companies plus others. It is important to note that the 
"fifth generation" project is an entirely distinct effort 
from the "supercomputer" project. The fifth generation 
project was formally initiated in April 1982 and will run 
approximately ten years. 

More immediate signs of the speed of Japanese 
development come from the very recent Hitachi and Fujitsu 
announcements of machines which are expected to compete 
directly with the yet to be announced Cray 2 computer which, 
when it arrives, will be the fastest U.S. machine and tue 
announcement by Hitachi of a 640 MFLOP array processor. 
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APPENDIX C 

Candidate Projects for Accelerated Development 

Several architecture candidates for accelerated 
development can be identified from among the recent research 
efforts into parallel computation. The following list is 
representative rather than exhaustive. The entries 
represent varying levels of maturity and differing degrees 
of completeness with respect to the spectrum of 
hardware/software and application problems. 

Blue CHip Project - Purdue 
Data Flow Project - MIT 
HEP2 - Denelcor 

High Speed Multiprocessor Project - Illinois 
Homogeneous Machine Project - Cal Tech 
Los Alamos PUPS Machine 
PASM Project - Purdue EE 

Systolic Arrays Project - CMU 

Texas Reconfigurable Array Computer (TRAC) - Texas 
Ultra Computer Project - NYU 

In addition, there is at least one candidate for an 

applications laboratory — the Purdue Center for Parallel 
and Vector Computing. 


The 

Blue CHiP Project, 

headed 

by 

Lawrence Snyder of 

Purdue 

(CS) , has as its 

focus 

the 

Configurable, Highly 

Parallel 

(CHiP) Computer. 

The 
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composed of a lattice of homogeneous computers each with 
local program and data memory that are connected together by 
programmable switches. By programming the switches the 
physical machine is configured to match the topology of 
algorithms. A preprototype CHiP computer is being built, a 
programming language and environment have been implemented, 
and the feasibility of wafer scale VLSI implementation has 
been worked out. 

Two projects to develop data flow computer architecture 

are in progress at MIT. In a data flow computer 
instructions are activated by data instead of an incremented 
program counter. The static architecture for a data flow 
supercomputer is being developed under Professor jack 
Dennis. Machine level code structures for several large 
application codes have been constructed by hand to prove 
performance potential, and an engineering model is in 
operation with eight processing units. The next phase 
requires development of custom LSI devices and an optimizing 
compiler for the VAL programming language. 

The tagged-token data flow architecture, being 
developed by MIT Professor Arvind, is a form of data flow 
multiprocessor using colored tokens to distinguish values 
associated with different procedure invocations. This 
architecture is ready for detailed simulation to determine 
good choices of architectural parameters and to evaluate 
program behavior and potential performance for realistic 
application codes. 
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HEP 1 is a commercially available, scalar MIMD computer 
system. HEP 2 is an upward compatible enhancement of the 
HEP 1 architecture which.will offer both increased speed and 
increased parallelism. HEP 2 will be very appropriate for 
general purpose scientific computing, and will be especially 
cost effective for those applications which exhibit 
parallelism of a non-vector kind. The logic and packaging 
technologies used in HEP 2 will be state-of-the-art in all 

respects. The projected completion date for HEP 2 is early 
1986. 


High Speed Multiprocessor Project — a University of 
Illinois group has. been engaged for the past ten years in 
automatically restructuring ordinary Fortran programs for 
high speed machines. Currently their system can do this 
effectively for parallel, pipeline and multiprocessor 
systems. These results have been demonstrated on a number 
of existing commercial machines. As a consequence of this 
work they are now designing a multiprocessor architecture 
that is aimed at providing high speed processing over a wide 
spectrum of applications. The design is based on 
measurements of over 1 000 application programs and as such 
is an example of a project that has developed an optimizing 
compiler hand in hand with a high speed architecture. 

The Homogeneous Machine Project, headed by Charles 

Seitz of Cal Tech, is focused around an array of processors 

for solving high energy physics codes. The processors are 

connected into a binary n-cube. a prototype built with 
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Intel 8086's, each with 65K of memory, 
construction. 


is under 


The Los Alamos Parallel Microprocessor Systems (Pups) 
is designed to accommodate up to sixteen computational 
processors and two communication processors. It can be 
configured as either a shared memory machine, a distributed 
machine, or as a collection of clusters of shared memory 
machines. It will support Fortran, have floating point, and 
a relatively large memory. 

The PASM Project, headed by H. J. Seigel of Purdue 
(EE), has as its focus the Partitionable Array SIMD/MIMD 
Computer. The PASM machine is composed of processing 
elements and common memory that can operate in SIMD mode or 
MIMD mode. The architecture is motivated by image 
processing tasks, and a design for an MC68000 based 
prototype has been completed. 


Systolic Array Project — These are computing 

structures which attain extreme efficiency by making use of 

designs in which data moves in very regular fashion through 

a sequence of processing nodes laid out in a dense pattern 

on the two-dimensional surface of a VLSI chip. Designs 

which keep all data paths short and process many items of 

data in parallel allow remarkable processing rates to be 

attained. Designs of this kind have been proposed by groups 

at Carnegie-Mellon University and elsewhere, and it is 

already clear that they can be very important for such 
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significant special applications as signal processing anti 
manipulation of matrices having a favorable band structure. 
The ability of this approach to produce "miracle chips" of 
great importance for new application areas is still far from 
exhausted, so that rigorous development of the systolic 
approach is appropriate. 

Texas Reconfigurable Array Computer (TRAC) — The TRAC 
system is based on coupling processors, memories and I/O's 
through a dynamically reconfigurable banyan network. TRAC 
implements multiple models of parallel computing. The TRAC 
project integrates consideration of hardware, software and 
applications of parallel computing. The TRAC architecture 
can be scaled up in number of processors and memory elements 
at a cost growth rate of n log2 n. TRAC is being developed 
as a laboratory for parallel computing. A 4 processor - 9 

memory configuration of TRAC is now operating and a Pascal 
compiler for programming of applications is available. 

The New York University Ultracomputer — This is a 

highly parallel MIMD machine which aims to combine hundreds 

or thousands of small, relatively conventional processing 

elements, all communicating with a large shared memory, to 

attain very high performance. In the present concept for 

this machine, processors communicate with memory through a 

very high bandwidth switching network which executes a few 

operations vital to ultraparallel inter—process 

synchronization in addition to its basic data-routing 

function. various advantageous operating systems software 
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structures have been worked out for this machine, as has a 
fairly detailed design for the switching chip control for 
its communication network concept. Various scientific 
application simulations carried out by the NYU group show 
that this machine can be programmed using relatively 
conventional techniques -{essentially in a slightly extended 
version of the widely used Fortran programming language) to 
attain high processor utilization. 

Purdue Center for Parallel and Vector Computing — The 
purpose of the center at Purdue University will be to 
advance the state of the art in the use of parallel and 

vector computers by engaging in research and development in 
areas such as the design and implementation of algorithms, 
performance analysis, modern language design and related 
computer development, appropriate software tools and the 
design and development of software modules. This work would 
be driven by specific application areas arising from the 
simulation of physical systems and would be accomplished by 
small multi-disciplinary teams consisting of Purdue staff 
and visitors from other universities, private industry and 
government laboratories. 
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Crystal Project - University of Wisconsin — The University of 
Wisconsin Crystal project, headed by D. Dewitt, has been funded by the 
NSF CER program to design and construct a multicomputer with 50 to 100 
nodes (processors). The processors are to be interconnected using 
broadband, frequency agile local network interfaces. Each processor 
will be a high performance 32 bit computer with approximately 1 megabyte 
of memory and floating point hardware. The total communications band¬ 
width is expected to be approximately 100 Mbits/sec. The multichannel 
capabilities of the frequency agile interfaces, along with the-Crystal 
support software, provide researchers a number of unique capabilities. 
First, the multicomputer can be divided into multiple partitions enabling 
researchers to share the facility in a manner analogous to a timesharing 
machine. In addition, the processors within a partition can utilize the 
frequency agile interfaces to efficiently emulate a number of interconnec¬ 
tion topologies. This will permit different groups of researchers to use 
the interconnection topology that is best suited for their application. 
Applications of interest include distributed operating systems, programming 
languages for distributed systems, tools for debugging distributed systems, 
multiprocessor database machines, and evaluation of alternative protocols 
for high performance local network communications. The system will support 
experimentation with parallel algorithms for solving computation intensive 
problems in the areas of mathematical programming, numerical analysis and 
computer vision. 
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APPENDIX D 

Applications and Benchmarks 

The list of applications given here are initial 
subjects for application development studies on the 
currently available parallel architectures. 

Monte Carlo techniques for simulating fusion processes. 
Specific programs in both time dependent and time 
independent models exist. 

Two— and three-dimensional hydrodynamic calculations in 
weapons design. 

Specific model computations include 

Par tide-in-Cel 1 for plasma models 

Adaptive Mesh Refinement for fluid processing 

These represent two algorithmic extremes and also the 
most current thinking, that covers both the physical model 
and how to fit onto parallel processors. 

Another class of computations is the Many Body problem 
used to study atomic and molecular interactions. 

Real time image processing is an integral component of 
inciny potential military applications. 

Two- and three-dimensional aerodynamics calculations 
for aircraft-like bodies. Techniques should include both 
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explicit and implicit 
vectorize well, the 
does not lend itself to 


methods; the former is known 
i- at ter has theoretical advantages 
parallelism as readily. 


to 

but 


Petroleum exploration and reservoir management are 
applications with very high potential economic payoffs. 
Reservoir management applications include elements of both 
fluid dynamics and heat transfer. Exploration studies deal 
with signal processing techniques. 


In all 

these 

examples 

the 

need is to 

interaction 

in 

order 

to 

inf1uence 


architectures. 


encourage early 
future computer 
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APPENDIX E 

Relationship to VHSIC 

The VHSIC program is focused on the development of very 
high performance device technology. The ultimate goal of 
this technology is to support military applications which 
require very high performance parallel computer systems. 
The purpose of the program proposed here is to accelerate 
the development of the architectures, systems and algorithms 
necessary to effectively apply this VHSIC technology. 

The VHSIC developed technology will begin to arrive 
next year. It is desirable to have in place a program of 
evaluation and utilization of this technology in appropriate 
architectures in order to provide feedback in the later 
stages of the VHSIC program. The program proposed in this 
report is a natural follow on to VHSIC. It has a broader 
spectrum of applications than only direct military 
applications such as real-time image processing. it extends 
to other crucial national security areas such as weapons 
development and energy production. It also includes 
economically critical applications such as weather modeling, 
geophysics, computer aided design and high performance 
robotics. 

The absence of such a program as is proposed here may 

lead to the Japanese being a principal beneficiary of the 

enhancement of high performance components generated by the 

VHSIC program through their effective system development of 
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APPENDIX F 

Relationship to Industry Programs 

The semiconductor industry through the Semiconductor 
Research Association (SRA) and the computer industry through 
the Microelectronics and Computer Technology Corporation 
(MCC) have begun to establish collaborative programs to 
support research in universities and to accelerate product 
development from university research programs in both the 
semiconductor and computer system fields. These initiatives 
by the computer vendors and semiconductor manufacturers 
strongly suggest that the role model projected here for 
cooperation between consortia of users, vendors and 
university research projects is one whose time has come and 
which will be accepted by industry as an effective means of 
accelerating product development in difficult and/or high 
risk areas. 

It is also possible that partial funding support can be 
obtained for these projects from these industrial consortia. 
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APPENDIX G 

Feasibility of Target Attainment 

It is the best reasoned judgement of computer 
architects, microelectronics researchers and software 
experts that the goal of a system approximately a thousand 
times faster than today's fastest computers in the early 
1990 time frame is viable. The requirements are a) 
component technology, b ) architectural and organizational 
structure, c) the software systems, and d) algorithms to 
exploit the architecture for significant applications. 


The VHSIC program is accelerating the development of 

very high performance component technology. There is need 

for a strong program in the development of packaging and 

cooling technologies in order to be able to exploit 

integrated circuit chips in very high performance computer 

systems, whether they be dedicated and special purpose 
systems or general purpose systems. 


A number e£ architectural concepts for exploiting 
parallelism which have promise of delivering very high 
performance computing systems have been defined and 
evaluated. These architectures will clearly be capable of 
delivering the required performance levels if they can be 
implemented in appropriate technology, be supported by 
appropriate peripherals, have the appropriate software 

available, and have the algorithms for application 
established. 
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The necessary peripherals such as large scale primary 
memories, high performance secondary memories, 
interconnection networks, etc. are the least developed of 
the architectural elements of a full system. It is clear 
that development paths do exist for these system elements. 

The development of basic concepts of parallel computing 
and the necessary software systems to support exploitation 
of these architectures is proceeding along with the 
development of the architectural concepts. A great deal of 
work is needed in this area, particularly with respect to 
applicability. 

The final requirement is the development of 
applications which are formulated in parallel concepts and 
which can exploit the very high performance parallel 
architectures. A very great deal of work is needed here. 
However, the initial work which has been done has been 
amazingly successful. 

In summary, in each element of the requirements for 
very high performance computer systems, there is substantial 
reason to believe that a sustained research and development 
program supported by effective engineering can approach the 
goal of 1000 times today's supercomputers. 
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appendix h 


Numeric and Mon-Numeric Computing: 

<3onship and Status 

Numeric and non-numeric computing have traditionally 
been regarded as different discipline areas. The two 
disciplines use different software, interfaces and 

algorithms even when sharing hardware environments. Numeric 
computing applications have traditionally driven the 

development of very high performance computer systems. This 
is because the problems to be solved are relatively well 
understood and thus attention could be focused on problem 
solution. While in the non-numeric areas (all basically some 
form of automation of the reasoning process,, relatively few 
problems have been sufficiently well understood to justify 
large scale application. Itose areas which have been 
systematically approached such as theorem proving and 

program verification havp c 

been .ound to require enormous 

computing resources for suhstanHai , . 

or substantial applications. Thus 

practical application .to non-numeric problems win probably 

require very high performance computer systems. 

The Japanese government and computer industry have 
recognised this state of affairs with the fifth Generation 
Computer Project which aims at producing "supercomputer 
SVlilir fpr artificial intelligence applications. 

The "Fifth Generation" and "Supercomputer" projects n£ 

Preserve the traditional separation of nulerio and 
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non-numeric computing. We believe that there are major 
overlaps in technology for both areas. Development of the 
parallel generation of "supercomputers'’ will have a major 
accelerating effect. on high performance systems for 
non-numeric applications. 

We further believe that there is a strong need for 
coupling automated reasoning with mathematical modeling and 
data base techniques to produce effective control processes 
for complex systems and for effective modeling of complex 
systems involving human and mechanical components. 

It Should be noted that the current generation of 
non-numeric applications are being developed on systems of 
power approximately one to two orders of magnitude below the 
current generation of supercomputers. There is need to 
consider scaling of non-numeric computing to the current 
generation of supercomputers. It may be desirable to 
organize a study panel to determine the national security 
and economic development impacts of non-numeric computing 
and the probable effectiveness of the entrepreneurial 
capital system of meeting these national security 
requirements as the research programs bring the concepts and 
applications to fruition. The Japanese government and 
computer industry have already established their commitment 

economic importance with the Fifth Generation project. 
It is worthy of note that there are now being marketed small 
scale systems specialized for non-numeric applications. 
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INFORMATION ON SELECTED FACILITIES 


AI-1 





> ? 


Partial Inventory and Announced Orders of Class VI Machines 


Country 


Si te 


Number 


Purpose 


Computer 


U.S. 


^ Los Alamos Nat. Lab. 
.a Lawrence Livermore 
Nat. Lab. 




v F 

A ' 

r a. : 

K C 


2 \ 


< 


i A 
I 


c? 


'I > 


1 . ‘ 4 2 - 

Germany 


4 

2 


Sandia Nat. Lab. 2 

KAPL 1 

- Bettis 1 

Kirtland Air Force Base 1 

National Center for 

Atmospheric Research 1 

*^NSA 2 

NASA-Ames 1 

NASA-Goddard 1 

NASA-Lewis 1 

FNOC-Monterey 1 

National Environmental 
Satellite Service (NOAA) 1 
Colorado State Univ. 1 

Univ. of Minnesota 1 

Geophysical Fluid 
Dynamics Laboratory 2 

Purdue University 1 

Univ. of Georgia 1 

CHEVRON 1 

Bell 1 

ARCO 1 

EXXON 1 

Grunman Corp. 1 

Westinghouse Corp. 1 

TEXACO 1 

SOHIO 1 

Digital Production, Inc. 1 

Boeing Computing Serv. 1 

Control Data Corp. 1 

United Information Serv. 1 

Max Planck 1 

Bochum 1 

PRAKLA 1 

Univ. of Karlsruhe 1 

Univ. of Stuttgart 1 

Deutch Forschungs und 
Versuchsanstalt fur 
Luft Raumfahrt 1 


Weapons Research 

Weapons Research 
Magnetic Fusion Energy 
Research 

Weapons Research 
Reactor Research 
Reactor Research 
Military 

Atmospheric Science 
Intelligence 
Aerodynamics 
Atmospheric Science 
Fluid Dynamics 
Oceanography 

Research 

Engineering Research 
Research 

Geophysics 

Research 

Research 

Petroleum 

Research 

Petroleum Engineering 

Petroleum Engineering 

Jet Engine Simulation 

Nuclear Power Plant Design 

Petroleum Engineering 

Petroleum Engineering 

Graphics 

Timesharing 

Timesharing 

Timesharing 

Research 

Research 

Research 

Research 

Research 


Aerospace Research 


Cray-1 

Cray-1 

Cray-1 
Cray-1 
Cyber 205 
Cyber 205 
Cray-1 

Cray-1 
Cray-1 
Cray-1 
Cyber 205 
Cray-1 
Cyber 205 

Cyber 205 
Cyber 205 
Cray-1 


Cyber 205 

Cyber 205 

Denelcor HEP 

Cray-1 

Cray-1 

Cray-1 

Cray-1 

Cray-1 

Cray-1 

Cyber 205 

Cyber 205 

Cray-1 

Cray-1 

Cyber 205 

Cray-1 

Cray-1 
Cyber 205 
Cyber 205 
Cyber 205 
Cray-1 


Cray-1 
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Partial Inventory and Announced Orders of Class VI Machines 


Country 

France 


Site 


Number 


Purpose 


GETIA 

Commissariat A'Lenergie 
Atomique 

Compagnie International 
De Services En 
Informatique 
Ecole Polytechnique 

England European Centre for 

Medium Range Forecasting 
Brit Met 
Daresbury 
AWRE Harwell 
Shell Oil, U.K. 

Univ. of London 
Univ. of Manchester 

Japan Mitsubishi 

Century Research 


1 

1 


Electric Power Institute 
Nuclear Energy 


Timesharing 

Research 


Weather 

Weather 

Physics Research 

Nuclear Energy, Weapons 

Petroleum 

Research 

Research 

Research 
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From W. F. Ballhaus, Jr. 
KASA/Anes Research Center 


August 13, 19 




For the past decade, NASA-Ames has conducted pioneering research in the 
rapidly advancing areas of computational fluid dynamics and computational 
aerodynamics as well as other aerospace disciplines. This research, con¬ 
ducted both on site and remotely at university and industrial sites, has 
been serviced by a continually expanding computational capability. Cur¬ 
rently this capability includes access to a Class VI computer housed on¬ 
site. NASA has plans to further augment its computational capability by 
means of the Numerical Aerodynamic Simulator (NAS), a project to-be con¬ 
ducted by the Ames Research Center. This facility will provide by 1988 a 
computation rate of a billion floating-point operations per second (sus¬ 


tained) with a memory sufficiently large to support this speed. The facili 
ty will provide interactive access, both remote and on-site, to a large? 
number of users from NASA, DOD, academia, and industry. It will be used to 
solve previously intractable problems of national concern in the aerospace 
disciplines of interest to the DOD and NASA. Limited operational capability 
is expected by 1985 with full service in 1988. The NAS facility will in¬ 
clude graphics and work stations, satellite and telecommunications inter¬ 
faces for remote access, a large data base mass store, and a fast network 
linking these elements with a high-speed computing engine. This year a new 
computer science effort has been initiated at Ames to seek innovative ways 
to apply advanced computational concepts to the solution of the Agency's 
technical problems. This activity will be complemented by a new institute 




at Ames under the auspices of the University Space Research Association. 


The Research Institute for Applications of Computer Science will be opei 


ating by early 1983. 
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NCAR COMPUTING CAPABILITIES AND SERVICES 
Walter Macintyre 

National Science *** WUST ° f - —rsed by 


° oJaanizaMnn?" ^ un j v f Sity ^search groups and other 

maior rlllTrl* ! iaentlf ^* develop and make accessible selected 

reauired bv the ervlces arid facilities of the outstanding quality 
required by the universities and NCAR for effective prooress in 

scientists in the universities and NCAR. 

° 2 f cooperation with universities and other organizations to Dlan 

l 1 " t 3 rest * and by the requirement for 

' - number S (3f institutions?^. ^ rUEt£ —» <* scientists fro. , 

1 innted^am^roodulated £ ^nVtur^osciences is 
both of NCAR 1 s missions require Ihl ?2t£ K ''f’ 1 C ° mpoters ’ He " ce 

*552 ?. C ??*S , ’ it * eS »* nepresent £ current 
wide variety of effective’services*’ rC Uecture ’ and sof tware, as well as a 

history, NCAR^as S p^oneered°i'n^akinq^dvanced^h d ! cades ag 3' Thr °ughout its 
readily accessible to the university-based and ncar"^ 6 ’ S0 ! tware and services 
NCAR Scientific Computing Division is wiSelJ ark^l^nL^ ^unity. The 
development of new capabilites fnr i-ho y acknowledged as a leader in the 

Committee on Oceans and Atmosphere for the President Vsripn*’ 0 ^ 1 Advisory 
Table 1 ), NCAR is shown to be’tSeon" flci? ?t? y o 6 o^'? 0 '' if 566 
university community on a first-priority basis overa11 


History of NCAR Computing Capacity 


19 «, T ^4S.’*SLS »■«. 

co~ B z 
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Table 1 

Simmary of existing large-scale operations in support of 
atmospheric modeling research, fiscal year I960* 


Aqency/Location 

Computers 

Users 

Percentage use 
for atmospheric 
research 


Type 


Percent 

Department of Conmerce 




NOAA 




GFDL, Princeton University 

r:/ASc 

in-house 

70-80 

Envlroneiental Research 
Laboratories, Boulder, Colo. 

CDC Cyber 170/750 

in-house 

50-60 

Suitland, Maryland 

(3) IBM 360/195 

in-house 

15 

Department of Defense 

AFGL, Bedford, Massachusetts 

COC 6600 

in-house 

n/a 

U.S. Army, unite Sands 

U 1180 

in-house 

n/a 

U.S. Navy, (FNOC), Monterey, 
California 

CDC 6500, 

Cyber 170/175. 

Cyber 170/720, 

Cyber 203 

in-house 

10 

NASA 



- 

G1SS. Ne» York. N.Y. 

IBM 360/195 

in-house 

75 

GSFC. Greenbelt, Maryland 

Amdahl 

(2) IBM 360/195 

in-house BOl 
universities 201 
in-house 901 

80 

20 

Hampton, Virginia 

COC Cyber 203, 

Cyber 170/173, 

(2) Cyber 173 
(2) Cyber 175 
(2) 6600 

in-house 951 
other 51 

20 

Pasadena, California 

(3) U 1108 

in-house 

1.5 

NSF 




NCAR, Boulder. Colo. 

Cray COC 7600 

NCAR 6 universities 100 

EPA 




Research Triangle Park, N.C. 

(2) U 1100, 

IBM 360/165 

in-house 

15 

DOE 




Argonne National Laboratory 
Argonne, Illinois 

IBM 370/195. 

IBM 370/75, 

IBM 370/50, (2) 30/33 

in-house 

5 

Brookhaven National Laboratory 
Upton, N.Y. 

COC 7600, (2) COC 6600 
DEQ PDP/10, Si^aa 7 

in-house 

3 

Battel le. Northwest Laboratory 
Richland, Wash. 

Not available 

in-house 

2 

Idaho Nat'l Engineering Lab. 

Idaho Falls, Idaho 

IBM 360/75, 

Cyber 76 

in-house 

n/a 

Los Alamos Scientific Lab. 

Los Alamos, N.H. 

(2) COC 6600. 

(A) COC 7600, Cray 1, 
(2) Cyber 73 

in-house 

O.A 

Lawrence Berkeley Laboratory 
Berkeley. California 

COC 7600, CDC 6600 

COC 6AM 

in-house 

1 

Lawrence Livermore Laboratory 
Livermore, California 

Cray 1. (A) COC 7600 
(2) Star/100, COC 66M 

in-house 

n/a 

Oak Ridge National Laboratory 

Oak Ridge, Tennessee 

IBM 360/75, P0P/10. 

SEL 8108 

in-house 

3 

Sandia Laboratories 

Albuquerque, N.M. 

U 1108.U1100-82, in-house 

(3) COC6600, CDC 6A00. 

COC 7600, Cyber 76, PDPIO 

0.5 

Savannah River Laboratory 

IBM 360/115 

in-house 

0.3 


Aiken, S.C. 

• from: A Review of Atmospheric Science Research Facilities . National Advisory Committee 
on Oceans and Atmosphere, Washington, June, 1981, p. 20. 
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History of Computing Output Units at NCAR 


OUTPUT 

UNITS 



Oac. Oac 

CDC 6600 CRAY- I 

< -^ ■<-► 


CDC 3600 

- ^ 


CDC 7600 


> 


Figure 1. Computer output units used at NCAR. The calculation of pow¬ 
er units is based on monthly hours of central processor (CP) time and 
peripheral (PP) time (on 7600), averaged over each year. When periph¬ 
eral time is not available, a factor is applied to CP time to estimate 
I/O production. If a machine is not in production for the complete 
first or last year, the data shown is the average for the months used 
during the given calendar year. 
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Year 

Late 1963 

December 1965 

July 1971 

June 1972 
February 1976 

February 1978 

March 1981 

September 1981 
October 1982 



Table 2 . Milestones of Major Computing Systems at NCAR 


Capability 


Characteristics 


CDC 3600 

CDC 6600 


CDC 7600 


RJE Capability 


Memory = 32,700 words, 48 bits each 
Overall speed = .06 of CDC 7600 

Memory = 65,000 words, 60 bits each 
Overall speed * 1.5 x CDC 3600 when new 
= 3.0 x CDC 3600 by 1968 
(after software system chan^ 
or 

= .2 of CDC 7600 

Memory = 65,500 x 60 bits small core 
= 512,000 x 60 bits large core 
Basic cycle time = 27 ns (speed 10-15 MIPS) 
Disks * 2 x 5.072 billion bits at first 
Early 1976 * 8 x 2.4 billion bits on di4|P 

Dialup from remote sites to Modcomo computer 
at NCAR Mesa Lab 


TBM Mass Store 
(used on-line 
Feb 1977) 


CRAY-1A 


IBM 4341 front 
end computer 


Oct 78 = 9300 active TBM volumes 
Aug 79 = 15000 active TBM volumes 
Aug 80 s 27000 active TBM volumes 
(Average volume size 72 million bits) 

Apr 82 = 43170 active TBM volumes 
(Total data volume 7.6 trillion bits) 

Memory = 1,048,000 x 64 bits 
Basic cycle time = 12.5 ns (and pipeline) 
(Speed 40-80 MIPS) 

Overall speed * 4.5 x CDC 7600 

Disks = 16 DD 19's x 2.4 billion bits each 

Used by SCD and other divisions for program 
job preparation. Also used for selected I/C 
tasks. Gives 6250 BPI tape capability for 
first time. Provides interactive access to 
the general user for the first time. 


NCAR internal Facilitates communication among the various 

network completed hardware components. 


Connection of NCAR 
system to a coirmon- 
carrier pocket 
switching network 


Gives all users of NCAR computers inexper 
interactive access to the system. 
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it possible for the science to address qualitatively different, previously 
intractable problems in many areas of atmospheric research. Table 3 shows 
the distribution of total NCAR CRAY computing resources to the uni versity-NCAR 
community by major disciplinary area during the period February 1978 to May 
1982. 


NCAR puts high importance on peripherals and systems to maximize the 
usefulness of the computer output and to minimize the time required of user 
scientists. It has the best developed and proven system for community 
participation in the process that governs the allocation of its resources 

NCAR's computing plans and priorities are set 
with active participation by the user community. In developing future plans 
for the facility, the assignment of first priority to the acquisition of 
additional main-frame power has been made with widespread community 
consultation and support. 

Current Capabilities of the NCAR Scientific Computing Division (SCD) 

}• Hardware capabilities . The major NCAR computing hardware 
capabilities- are a CRAY-lA, capable of executing 80 million instructions per 
second for very high speed computations; a Control Data Corporation (CDC) 7600 
for fast data analysis and file manipulation; an IBM 4341 computer system for 
providing interactive access to the major computing engines; an AMPEX Terabit 
Memory System (TBM), which provides mass storage for archived data as well as 
significant on-line data storage; and a remote job entry system that 
communicates with 75 locations across the country. The SCD also has a DICOMED 
graphics systems, a Network Systems Corporation (NSC) high-speed data network 
to connect the various systems, and a GANDALF port contention device which 
permits selective use access to the various machines. 

2 - Services . The services provided to the community by the NCAR SCD 
include the following: 

a. Data Support. The Scientific Computing Division's Data Support 
Section maintains a large archive of computer-readable research data and 
provides assistance to users in locating data appropriate to their research 
needs, interfacing their programs with the data sets, and accessing utility 
routines for manipulation of the data. Users can access the data from remote 
terminals, in addition to using the data at NCAR or receiving tapes. This 
group has achieved a worldwide reputation in acquiring, formatting, updating, 
and making accessible atmospheric observational data sets. 

The Data Support Section maintains many large sets of analyzed 
grid data and observed data from the National Meteorological Center, the 
National Climatic Center, the U.S. Navy, and the U.S. Air Force. Other 
countries and laboratories also provide data. Supporting data such as land 
elevation and ocean depth are included. The archives are largely describd in 
Data Sets for Meterological Research , by Jenne (NCAR-TN/IA-111, 1975). 

b. User Services. The User Services Section offers consulting 
services to users, provides information on all services and operational 
procedures of the division, and provides software libraries of numerical and 
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Table 3 

TOTAL CCU USE BY AREA OF INTEREST 
c ebruary 1, 1978 through July 31, 1982 


AREA OF INTEREST 

CCU's 

% TOTAL 

1 . 

Cloud Physics 

1,225.7 

12 

2. 

Weather Prediction 

2,147.2 

21 

3. 

Solar Physics 

1,213.7 

12 

4. 

Chemistry and 

Upper Atmosphere 

949.2 

9 

5. 

Climate 

2,979.7 

29 

6. 

Oceanography 

765.8 

7 

7. 

Basic Fluid Dynamics 
& Miscellaneous 

1,092.4 

10 


Total: 

10,373.7 

100 
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utility tools. This section provides an effective interface to the hardware 
and operating system software for user scientists. 

Within the User Services Section, the Communication group 
manages the current communication links for remote job entry (RJE) and 
interactive access. The Library Group manages and distributes the larqe 
nunerical library (4700 routines) already available from SCO and develops new 
tools that facilitate program development and maintenance in a multi-machine 
environment. The User Interfaces Group provides documentation and 
consultation services and continuously improves methods of access to the 
facilities. The Multi-User Software Group is now completing a multi-year 
project for GENPR02, a generalized software package for signal processing and 
data analysis. 


c. Systems. The Systems Section develops network, communication 
and mass storage systems, as well as maintaining operating systems and their 
related software and language compilers supplied by vendors for several types 
of computers, ranging from small- to large-scale machines. 

t , The highest priority is the maintenance of the integrity of the 

current operating systems, including the software running the internal NCAR 
network. The staff are involved in the development of interface software and 
non-vendor provided systems software as well as the provision of data 
maridgement software. They assist with hardware and software planninq and 
acquisitions. 3 

<■ . . ‘ . Operations and Maintenance. The Operations and Maintenance 

Section operates and maintains the hardware systems of the facility and 
provides digital data library services, statistics on system use, and 
microfilm/microfiche and movie production. 

e. Advanced Methods. The activities of the Advanced Methods Group 
include research, consulting and the production of advanced mathematical 
software in such areas as thermospheric physics, computational fluid dynamics 
and spherical vector harmonics. 

. 3 - Algorithm a nd Software Development . The NCAR computinq staff and 
various users have contributed to the development of algorithms and software 
that are valuable in many areas of science. Examples are vectorized fast 
Fourier transform software, non-linear second-order partial differential 
equation algorithms and software, and the NCAR graphics package. This work 
continues and is effective disseminated throughout the scientific community. 

Figure 2 gives the current organization and staff levels of the NCAR 
Scientific Computing Division. 

Allocation Procedures 


Allocations of NCAR computer resources are governed by the UCAR Board of 
Trustee policy, which requires that 45X of NCAR computing resources be 
avaiiable to university users, 45% to NCAR research and 10% to joint 
NCAR-university projects. Use of the computers by postdoctoral and other 
long-term visitors and processing of data from aircraft, radars etc i< 
counted in the NCAR allocation, unless the investigator involved has applied 
for resources in the university category. This seldom occurs. 
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SCO ORGANIZATION CHART 




rsj 


1) 

2 ) 

3) 

4) 

5) 

6 ) 


Computer OpentIons - Operates and maintains all SCO computers, peripherals, and related equipment; accounts for the usage of this eoulpwnt; 
processes all film and flche output. 


Data Support - Acquires, prepares, and maintains archives of meteorological, oceanographic, and other necessary data; provides counseling services 
regarding these archives; fulfills requests for data sets for use elsewhere. 


Systems Programming - Maintains the operating systems on all SCO computers; develops Interfaces to these systems for user programs. 

Advanced Hethods - Research, consulting, and software development for problem solving In scientific computing. 

User Services - Provides the services required to allow users access to SCO computers, documents user Interfaces to SCO computers, consults with 
users on problems, trains users, develops and maintains multiuser software, and acquires and maintains program libraries for general use. 


Di vision Director's Office - Overall responsibility for delivering large-scale computing services to users; for allocation of confuting resources 
to non-NCAR users; for administering the activities of SCO Including planning, budgeting, and general supervision; for delivering support 
services to division personnel. * rr ^ 
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rnr ,!nH' NCAR rec iuests for a total of more than five hours on the CRAY-1 or 
CDC 7600 are reviewed by the SCO Advisory Panel after preliminary review by 
JlJJ n° re 1ndlvidual ^viewers. The Panel meets twice a year, in the spring 
nf d r? T members are selected from the community at large on the basis 

rIiatoH b Jl S ^ d co "] pete !? be 1n the atmospheric sciences, computer science, and 
outside NCAR di ‘ N0rma y ’ ^ee-quarters of the Panel members are from 

The panel assesses as the merit of proposals for che use of the SCD 

nn C ; p f reC0 TJ S acti0n With respect t0 a Prospective user‘s request 

on the basTs of scientific merit, computational effectiveness and need*. The 
following specific questions are asked about each proposal: 

o What contribution is the project likely to make to the advancement 
of the atmospheric sciences? 

o Is the work original? 

Are the scientific approaches and techniques appropriate? 

Will the project make efficient use of computing resources 7 Will it 
be I/O bound? Are current mathematical and numerical methods used? 

Are appropriate algorithms employed? 

th.t ?! Panel ^ ay re ^ omniend that the request for resources be fully granted 
that only a portion of the request be granted (which will mean the proiect ’ 
must be scaled back or stretched out in time), or on occasion thaAhe 
request be denied. In cases where a request for computing at NCAR is an 

KTaf! a PrOPOSa ' t0 NSF - NCAR process Jitf 

Requests for university use of less than 5 hours of CRAY-1 or CDC 7600 
5 valu - ated b r t^ Director of the Scientific Computing Division with 
cofnJ^itv fr °Th SC1entl [ 1C / eV1ewerS selected fr om the NCAR staff or external h 
Advisory Panll ^ °" S are reviewed as a group twice a year by the SCD 

NCAR use of the computer is allocated by the Director of NCAR as nart 
the overall NCAR budget and resource allocation systems. The Director 
presents and defends his proposals for NCAR programs and associated resourrp 
allocations before the UCAR Board of Trustees and its Budget and Proqram 
Committee. The efficient use of resources by NCAR staff is also reviewed 
s?wi bhe Scientific Programs Evaluation Committee (SPEC) process in which 
scientists drawn from the comnunity at large review a 1 aspects of NCAR 
programs on a triennial basis. aspect or hlak 

The joint use allocation is also administered by the Director of NCAR. 
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MAGNETIC FUSION ENERGY AND COMPUTERS* 


John Killeen 


National Magnetic Fusion Energy Computer Center 
Lawrence Livermore National Laboratory 
Livermore, California 94550 


The application of computers to magnetic fusion energy research is essential. In 
the last several years the use of computers in the numerical modeling of fusion 
systems has increased substantially. There are several categories of computer models 
used to study the physics of magnetically confined plasmas. A comparable number of 
types of models for engineering studies are also in use. To meet the needs of the 
fusion program, the National Magnetic Fusion Energy Computer Center has been 
established at the Lawrence Livermore National Laboratory. A large central computing 
facility is linked to smaller computer centers at each of the major MFE laboratories 
by a communication network. In addition to providing cost effective computing 
services, the NMFECC environment stimulates collaboration and the sharing of computer 
codes among the various fusion research groups. 


INTRODUCTION 

In June 1973 an Ad Hoc Panel on the Application 
of Computers to Controlled Thermonuclear 
Researchl was convened at the USAEC in order 


6. Multi-species Fokker-Planck codes 

7. Hybrid codes 

Engineering models needed in fusion reactor 
design studies include 


(a) to survey and summarize the existing 
level-of-effort in the application of computers 
to CTR. 

(b) to identify important CTR physics and 
engineering questions that are or may be soluble 
by the use of computers, and to evaluate the 
benefits that would accrue to the CTR program 

if such solutions were obtained, and 

(c) to survey, summarize and evaluate the 
status of' present and anticipated computer tech¬ 
nology for the purpose of accurately forecasting 
the type, size, scope, and composition of a 
facility that would realize the benefits 
identified in (b) above, and the lead time 
necessary for assembling it. 

The wide range of the questions posed to the 
panel dictated that its composition be as 
diverse as the issues it was asked to address. 
Therefore, besides AEC Headquarters personnel, 
the panel had three different groups of parti¬ 
cipants—plasma physicists, plasma engineers, 
and computer scientists—who were organized 
along functional lines into three sub-panels. 

A recommendation of the Ad Hoc Panel was a 
significant expansion in the development and 
use of computer models in the fusion program. 

The following plasma physics models were identi¬ 
fied and their importance to the fusion program 
discussed.^ 

1. Time-dependent magnetohydrodynamics 

2. Plasma transport in a magnetic field 

3. MHD and guiding-center equilibria 

4. MHD stability of confinement systems 

5. Vlasov and particle models 


1. Plasma engineering-burning plasma 
dynamics 

2. Nucleonics 

3. Mechanical design 

4. Magnetic field analysis 

5. Systems studies 

6. Thermal hydraulics 

7. Tritium handling 

8. Safety and environmental studies 

Another recommendation of the Ad Hoc Panel was 
the establishment of a computing facility dedi¬ 
cated to the magnetic fusion program. The 
National Magnetic Fusion Energy Computer Centei 
(NMFECC) was organized in 1974 at Lawrence 
Livermore Laboratory and service began in late 

1975. From the above lists of computer models 
in plasma physics and engineering it is clear 
that the fusion program requires the most 
advanced scientific computer available. In 
September 1975, NMFECC installed a new CDC 760C 
dedicated to fusion physics calculations. 

A review of requirements at the National MFE 
Computer Center was conducted in the spring of 

1976. The results of this review led to the 
procurement of the CRAY 1, the most advanced 
scientific computer available, in the spring of 
1978. The utilization of this computer in the 
MFE program has been very successful.2 

A new study of computer requirements for th^^ 
MFE program was conducted during 1979.2 Thq^^T 
MFE program has grown considerably since 19liP 
and major directions have emerged, e.g., TFTR, 
MFTF, and the proposed Fusion Engineering Devlc 
(FED). The computing requirements are 
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necessarily much larger and the models more 
varied and demanding.2 The increased emphasis 
on fusion technology has led to a substantial 
growth in the use of the NMFECC for engineering 
calculations. The 1979 study 2 led to the pro¬ 
curement by the NMFECC of a second CRAY 1 
computer in September 1981. The study 2 also 
recommended the addition of a more advanced 
computer during 1984. 

THE NATIONAL MFE COMPUTER NETWORK 

The purpose of the MFE Computer Network is to 
provide to all fusion researchers the full range 
of available computational power in the most 
efficient and cost effective manner. This is 
achieved by using a network of computers of 
different capability tied together and to the 
users via dedicated data lines and dial up 
telephone lines. The existence of this nation¬ 
wide computer network allows projects to be 
sited anywhere in the country without regard to 
local computer availability, and therefore 
increases enormously the flexibility of the 
fusion program. 

The Center began first operations using a CDC 
6600 loaned by the Lawrence Livermore National 
Laboratory Computer Center (LCC). In September 
1975 the Center installed its own CDC 7600 
computer, and by March 1976 significant trans¬ 
mission over the dedicated Data Communications 
Network was taking place. By that time the new 
CDC 7600 was saturated with a calculation work- ' 
luad which had been implemented, until then, 
solely by dial-up telephone communications 
between the Center and the user community. 

Levels of Computer Capability in NMFECC 

The concept of the NMFECC is that different 
levels of computer capability are provided at 
the various remote locations according to 
research priorities and anticipated computa¬ 
tional demand. At the national center, provid¬ 
ing high level capability to the entire 
community, is the original CDC 7600 plus two 
high-speed CRAY 1 computers with one and two 
million words of memory, respectively. 

Additional equipment at the national center 
includi”. processors and other ADP equipment for 
ronmunirations, file management, and data 
storage. (Figure 1) 

At the next level of capability are User Service 
Centers (USC's): DEC-10 computer systems with 
direct high-speed access to the national center 
through PDP-11/40 remote communications control 
processors. There are now five operational USCa 
(Figure 2) in the field located at Princeton 
Plasma Physics Laboratory (l’PPL), the Los Alamos 
National Laboratory (LANL), the Oak Ridge 
National Laboratory (ORNL), General Atomic (CA), 
and LLNL (for the mirror confinement program). 

A sixth USC, used in center operations, is 
located at the NMFECC itself. 


NMFECC HARDWARE CONFIGURATION 



A third level of capability is provided through 
the Network Access Port (NAP). MFECC designed 
the NAP to permit remote computers to be connects 
ed to the MFE network as remote hosts. There 
are currently five NAPs installed, all of them 
connecting VAX 11 series computers into the 
network. 

A fourth level of capability is provided by 
Remote User Service Stations (RUSS) at selected 
MFE research sites. RUSS stations are current¬ 
ly installed at 18 remote locations (Figure 2). 
RUSS stations provide users with the capability 
of printing output files locally on a 1000 line/ 
minute printer and act as a terminal concentra¬ 
tor for up to 16 interactive terminal users. 

RUSS stations are connected to the nearest MFE- 
NETWORK communications processor over a 4800 
baud dedicated line. (Figure 2) 


NATIONAL MFE NETWORK 1982 




Figure 2 
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A fifth level of capability is dial-up access. 
MFE researchers at some 35 localities access the 
center using computer terminals equipped with 
acoustic modems or couplers. 

Data-Communicatlons Systems 

Data Communications service to the National MFE 
Computer Center is provided on a 24 hours/7 day 
basis. Three types of service are provided to 
NMFECC users as outlined below: 

1. Wide band (56 KB/sec) Satellite Network 
Service. Users at Major USC*s on the MFE 
net may log on to their local DEC-10 system and 
interact with the computing resources at the 
Central facility in Livermore. Currently four 
major network satellite links are in service 
from LLNL to Princeton, N.J., Oak Ridge, Tenn., 
Los Alamos, N.M., and San Diego, Calif. These 
are dedicated dual channels and modems which are 
connected to LLNL owned communication control 
processors (DEC 11/40's). (Figure 3) 

MFENET TYPICAL SITE CONFIGURATIONS 

Tops 10 



2. Dedicated 4800 Baud Service. Remote User 
Service Stations on the MFE Net are served by 
dedicated leased 4800 baud lines which terminate 
either at the Center (LLNL) or at the nearest 
MFE Communications Control Processor. Location s 
served by 4800 baud dedicated service are shown 
in Figure 2. 

3. Dial Up Service. Users not at major 
fusion laboratories may dial-up the Center using 
one of the following services: 


(b) ARPANET ACCESS—NMFECC users with a ^fe| 
to an ARPA IMP may use the ARPANET as a m^p 
communicating with the Center. 

(c) DIRECT DIAL COMMERCIAL or FTS—Thirty 3C 
b aud and ten 1200 baud ports are available throw 
auto-answering modems which connect dial up 
users to MFECC’s terminal concentrator. 

NMFECC COMPUTING ENVIRONMENT 

The NMFECC computing environment reflects the 
needs of computer users in the Magnetic Fusior 
Energy research community. Both interactive 
timesharing and batch processing are available 
A s umma ry of some service follows: 

Timesharing Services 

The fusion community has always found that 
interactive computing, even with the largest 
codes, is by far the most efficient use of 
physicists efforts. The 5Z overhead in swap¬ 
ping codes in and out of the machines provides 
fast debugging, immediate turn around on key 
results, and the capability to interact with 
codes which need user control. The Livermore 
Time Sharing System (LTSS) developed for the 
CDC 7600 by the LCC was adapted by the NMFECC 
for the CRAY 1 computer in about six months. 
CTSS was available as the first CRAY 1 wa^Bj 
vered and the final bugs-were removed wit^^P 
a couple of months. CTSS is supported by 
libraries of FORTRAN callable subroutines whi< 
enable a user to issue almost every system ca] 
giving access to every part of the hardware, 
typical physics code can be run from a termini 
display graphics as it runs, be interrupted oi 
interrogated at any time. The ability to stai 
or stop a code at any point and inspect the 
results provides debugging at least 100 times 
faster than older methods. 

File Storage Services 

NMFECC has designed a multi-level file storagi 
system called FILEM. FILEM is a highly versa¬ 
tile system which allows users to store and 
retrieve programs and data files, in the centr< 
computing facility at Livermore for an indef¬ 
inite period of time. FILEM has been designei 
to acconsnodate the needs of users at remote 
sites. The CDC 7600 has been programed to 
assume virtually all of the tasks associated 
with file custodianship including indexing, 
storage, retrieval, and efficient management 1 
the file storage media. The NMFECC file 
storage media currently consist of three leve 
of storage (Figure 4). 


(a) TYMNET—A Tymnet owned CP-16A/1200 
processor is installed at the NMFECC in Liver¬ 
more. NMFECC users have access to the six 
unlimited 300 baud ports and two unlimited 1200 
baud ports. TYMNET calls are also routed 
through the LLNL owned terminal concentrator. 


User Services 

It is the policy of NMFECC to make all 
documentation available on line so that users 
may provide themselves with up-to-date system 
documentation by simply printing out the docu 
ment at their local printer or terminal. Any 
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part of any document may be displayed on remote 
terminals and the routine DOCUMENT is capable of 
scanning text for the user to locate a specific 
topic of interest.. NMFECC has provided the user 
with two routines called MAIL and NEWS which 
allow users to send a message or question to any 
other user on the NMFECC network. NEWS and MAIL 
are also commonly used by users to ask NMFECC 
staff about specific problems they have encount¬ 
ered. NMFECC systems programmers and docemen- 
tarians use NEWS to broadcast all system or 
documentation changes. Users who are unable to 
solve a computational problem by consulting 
DOCUMENT or inquiring through NEWS may seek 
assistance from the software consultation staff 
at che center. Depending on the user's needs 
the staff may diagnose problems, recommend sol¬ 
utions, and follow through to insure that a 
satisfactory solution is realized. Specialized 
assistance in the areas of mathematical librar¬ 
ies, graphics, engineering analysis and symbolic 
manipulation is also available through the 
Center. 


NMFECC CENTRAL STORAGE 


CRAY-1S | CRAyT) ) 

' cv 


Destaqmg 



CDC-7600 


Staging disk 

Capacity - 2400 Mbyte 
Access - 35 millisec 
File life - 1 week 

Mass storage 

Capacity - 62500 Mbyte 
Access - 10 seconds 
File life - 12 weeks 


*) Automated tape library 

__on's/- .. Hanaritv - ^OQ^flO MKwt 


Memorex 3226 tapes Capacity - 209500 Mbyte 
Access - 2 minutes 
File life - archival 

Figure 4 

LIBRIS Computational Plasma Phvsics and 
Engineering Data Base 

NMFECC 1 s policy is to encourage the exchange of 
information about application codes within the 
MrE computational community. To facilitate 
sharing, a common data base of computational 
physics and engineering codes is available 
through a routine called LIBRIS, which allows 
any user to abstract a code that he wants to 
make available to the MFE community or to inter¬ 
actively interrogate the abstracts in the LIBRIS 
data base. 


NMFECC PLANNING PROCESS 


The need for a more advanced computer in 1984 
should properly be discussed from two perspec¬ 
tives. The first perspective has to do with 
capability and can be discussed in the context 
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of the following question: Are there important 
fusion problems that cannot be solved with; 

' todays most capable computers? The second per¬ 
spective has to do with capacity and suggests 
the question: With two CRAY machines in opera¬ 
tion does NMFECC still fall short of the 
projected requirements of the fusion program? 
The answer to both questions is yes. 

The Need for More Powerful Computers 

The central focus of magnetic fusion theory is 
the behavior of plasma as it interacts with 
magnetic fields. The growing body of physics 
theory that pertains to this interaction is 
extremely complex. Computers such as the CRAY 1 
have permitted code designers to simulate plasna 
interactions more accurately, but improved 
models are needed and the improvements will 
require machines more capable than a CRAY-1S. 

It should be emphasized that increases in memory 
size should be matched by increases in CPU 
cycle time. Expanding the parameters of a 
problem to use a larger memory size might other¬ 
wise result ir, unreasonably long run times to 
obtain a result. Thus we look to a machine 
which will demonstrate significant improvements 
in both memory size and execution speed over 
the CRAY-1S. 


We can examine the need for more powerful com¬ 
puters with respect to some of the major types 
of codes which are used in the MFE program. 


Normal Mode Codes — One approach to 
the determination of MHD behavior is the calcu¬ 
lation of linear normal modes; that is, natural 
modes of oscillation of small perturbations 
away from equilibrium. This has been carried 
out in very detailed fashion by the PEST and 
ERATO codes, which calculate an equilibrium and 
then obtain the structure and frequencies of 
these normal modes. These codes have made vital 
contributions to the understanding of the dang¬ 
erous kink and ballooning instabilities, as 
well as other MHD modes in tokamaks. This has 
permitted the much more accurate determination 
of limits on the plasma beta, the ratio of the 
plasma pressure to the magnetic field pressure. 
Both codes run approximately ten times faster 
on the CRAY 1 than on the CDC 7600. This sub¬ 
stantial decrease in running time results not 
only from the fact chat the CRAY 1 is a much 
faster machine, but also because its fast 
access memory is substantially larger, thereby 
permitting more efficient coding. Even so 
both PEST and ERATO and other codes of their 
type require a great deal of computer time 

fw Ch J“ P “ Cer run takes abouc chi «y minutes on 
the CRAY 1; several runs are required to analyze 
one equilibrium; and hundreds are required to 
discover trends. Such stability calculations 
will be vital in the investigation of future 
experiments and reactor configurations. 


Time-dependent MHD Codes - The other 
technique for determining plasma instabilities 
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along with their growth rates is through the 
solution of the time dependent MHD equations of 
motion. The full set of MHD equations comprise 
a coupled system of eight nonlinear partial 
differential equations, the solution of which is 
a formidable task on any computer system. In 
order to make these computations tractable, 
approximations have often been made, including 
reduction in dimensionality, linearization, re¬ 
striction to a particular geometry,*ordering, or 
regime, and the assumption of no transport or 
resistivity. To explain complex phenomena such 
as the major disruption in a tokamak requires a 
three dimensional resistive code. It is of 
utmost importance to understand the major dis¬ 
ruption, for if it occurs it will cause the 
tokamak walls to vaporize. 

The rate of progress of resistive MHD calcula¬ 
tions in tokamaks has been strongly dependent 
on two factors: advances in numerical techniques 
and increases in computer capability. These 
calculations have produced a succession of 
results which have given increasingly detailed 
comparison with tokamak experiments. However, 
ro obtain results on a CRAY 1, it has been nec¬ 
essary to impose numerous simplifications which, 
if removed, might materially increase our under¬ 
standing of fusion devices. We expect that 
further advances in numerical techniques will 
be hard to achieve and will offer even smaller 
speed advantages. Thus, further progress in 
this area is likely to be strongly coupled to 
computer capability. 

The theory is at a stage now where one can con¬ 
struct a computer program to include "exact" 
treatment of tokamak geometry and pressure 
effects with a realistic level of resistivity 
and some kinetic effects. However, it is doubt¬ 
ful that the CRAY 1 has the capability to run 
such cases. It is even more doubtful that such 
calculations can be made for stellerator config¬ 
urations. The primary limitation is CPU speed. 

A secondary consideration is increased fast 
memory size. 


The recent advance in three-dimensional MHD 
calculations for tokamaks has depended crucially 
on obtaining a reduced set of equations by 
expanding the original MHD equations in a small 
parameter, which is on the order of the inverse 
aspect ratio. This is possible because of the 
strong and almost uniform toroidal magnetic 
field in tokamaks. Since the field components 
in the Reversed Field Pinch (RPP) are all of the 
same order, and since these devices possess 
finite beta, there exists no universally small 
parameter in which to expand the basic equations. 
Additionally, the computational speed of the 
codes based on the tokamak reduced equations is 
greatly enhanced by the assumption of incom¬ 
pressibility, which eliminates the compress- 
ional Alfve'n wave. Because of the strong field 
in a tokamak, the fastest remaining mode evolves 
on a time scale on the order of the major cir¬ 
cumference divided by the Alfve'n velocity. This 
time scale may be more than an order of 


magnitude longer than that of the compressioi 
Alfven wave. In the RFP, on the other hjtfjj^k 
even the assumption of incompressibilit}MBB 
not provide much of an advantage, since 
shear Alfven wave propagating near the field 
reversal point evolves on a time scale on th 
order of the minor circumference divided by 
Alfven velocity, a reduction of a factor of 
only 2 or 3 from that of the compressional 
Alfven wave. Thus the next meaningful step 
MHD simulations for the RFP will tax even th 
next generation of computers. 

To make these three-dimensional codes applic 
to more general geometries (e.g., stellarato 
and to simultaneously include enough effects 
ensure a complete description of the importa 
physics effects (e.g., parallel heat transpo 
compressibility, finite larmor radius effect 
and smaller values of resistivity) will requ 
a machine with about 100 times the CPU speed 
the CRAY 1 in order to keep the run times ab 
the same i.e., tens of hours. Finally, an 
increase of a factor of 10 in memory size wc 
allow a factor of 2 increase in each direct! 
of a three-dimensional calculation, while kc 
ing the entire code resident in fast memory. 
This increase would provide a significant 
improvement in resolution. 

( c ) Particle and Hybrid Codes - In many c< 
fluid models are not adequate to descrihgU 
behavior, for it is necessary to considMp 
scopic effects, i.e., the effects of the^Sl: 
particles are distributed in velocity. Nunn 
cally this is most often accomplished throuj 
particle codes. Fully nonlinear kinetic ioi 
and electron simulations in 2-D"Cartesian 
geometry were carried out over the last dec: 
In the past, Cartesian geometry was not a nu 
physics limitation even with the obvious cy. 
drical and toroidal nature of experiments, 
because these models necessarily dealt with 
length and time scales on the order of the 
electron gyroradius and plasma oscillation 
period for stability. Resolving such lengtl 
and time scales meant that any realistic ma< 
scopic dimension could be considered infini 
in light of the huge number of computations 
time steps required for any information to 
travel such a distance. Such models are pr 
marily useful for plasma transport studies, 
which are made computationally accessible b 
studying relatively sharp gradients. Even 
present 2-D computational methods still req 
an unrealistically small mass ratio M^/M^ a 
other artifical compression of disparate ti 
scales to retain acceptable run times. Impl 
and orbit-averaged methods improve the situ) 
somewhat, but routine 3-D simulation with r 
istic parameters is simply not practical wi 
present computational resources. In addd^l 
with the increase of grid resolution al^Ej 
improved computers and methodology, the^K 
of particle simulations has grown to encomp 
nonlocal effects and more realistic geometi 
This further adds to the complexity of code 
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And has lead co renewed demand for more memory 
and speed of the computer. 

On present computers, large scale particle sim¬ 
ulations in 2-1/2D and 3D are mainly limited by 
the size of the maximum fast memory the CRAY 1 
can handle (of the order of 1 M words, or 2 M 
for the CRAY-IS). With necessary diagnostics 
this amount roughly corresponds to two-dimen¬ 
sional grids of 128 x 32 or 64 x 64 for electro¬ 
magnetic particle codes and to a three-dimension¬ 
al grid of 32 x 16 x 8 for MHD particle cedes. 
For example, in order to have relevant mode- 
conversion physics for Ion Cyclotron Resonance 
Heating (ICRH) in tokamak geometry, a minimum 
grid of 128 x 128 was necessary. 

An enhanced Class VI computer with 2.5 times 
more memory and speed than a CRAY-1S will permit 
a grid of 128 x 64 and perhaps 128 x 128 in an 
electromagnetic particle code and use more real¬ 
istic parameters. Experimentally relevant 
physics problems in magnetic confinement have 
important three-dimensional aspects, such as in 
the multiple-helicitv interaction of collision¬ 
less.tearing modes and in the drift wave turbu¬ 
lence in sheared magnetic "fields; the - increased 
memory and speed will increase the practicality 
of 3-D simulations. It is, nevertheless, clear 
that such an increase in memory will not be 
enough. The advent of a Class VII computer with 
memory of the order of 30 M words, for example, 
will be able to tackle a 512 x 128 system in 
electromagnetic codes, corresponding to a plasma 
of the size of several tens to a hundred 
collisinnless skin depths. 

In addition to this, one should consider the 
acquisition of a large fast solid-state peri¬ 
pheral memory. This will allow users to double 
or triple-buffer the grid and particles without 
resorting to much slower discs. It would be 
helpful to have a solid-state memory of around 
a few hundred million words for large scale 2D 
and 3D calculations. When a large fast solid- 
state peripheral memory is attached to the fast 
core, it is very important to have fast, 
efficient, large scale I/O between the core and 
memory. It may be preferable to arrange the 
solid-state memory as virtual memory with some 
paging capabilities. 

Particle-fluid hybrid models have become impor¬ 
tant in the last five years. A typical hybrid 
model represents the ion components as kinetic 
species and the electrons as a fluid in order 
to eliminate some or all fast electron 
frequencies and short length scales. Without 
these electron-imposed limitations, the kinetic 
ion effects can be modeled on macroscopic, 
almost MHD, time and length scales-making 
experimental relevance much easier to establish. 
The ion temperature gradient drift instability 
in a tokamak was studied with such a model. 
Sometimes, without sacrificing the parallel 
dynamics of electrons, the guiding-center parti¬ 
cle model is used. This method has been 
actively used particularly for tokamak plasmas. 
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Recent progress with hybrid models is impress¬ 
ive but is still quite computationally expensive 
(typically taking roughly two to four times 
more CRAY CPU time than does an MHD code of 
' equal dimensionality). Further, 2D meshes of 
size 128 x 128 with 20 particles per cell lead 
to memory requirements of order 2 x 10*> words. 
Results obtainable with present 2-D codes, as 
well as progress on 3-D codes, are presently 
hampered by lack of CPU speed and memory capa¬ 
city. For example, typical 2D quasi-neutral 
hybrid simulations of rotational instabilities 
in the Field Reversed Experiment at Los Alamos 
use 50,000 particles, require approximately 3 
hours of time on the CRAY 1, and use half of 
the CRAY 1 active roemorv. 

In order to resolve ion spin-up effects in 
rotational instabilities of compact toroids, it 
would require at least 10 hours of CPU time oh 
the CRAY 1 for a simulation run with a present 
2D hybrid code. It would also be deslrahle to 
increase the number of particles used in simu¬ 
lations in order to more properly represent the 
ion velocity distribution in low density 
regions; however, this is not practical on the 
CRAY 1, without resorting to disks and buffer¬ 
ing and hence motivates the use of a Class VII 
computer. 

Another new particle simulation technique that 
has led to much more realistic simulation of 
fusion experiments on transport time scales is 
orbit averaging. In orbit-averaged simulations, 
time-splitting has been combined with temporal 
averaging to allow the self-consistent solution 
of Maxwell's equations on a slow time scale 
using a long time step, but the particle 
dynamics are followed on the natural time scale 
of the orbit. Orbit averaging in a two- 
dimensional, magneto-inductive algorithm has 
been successful. A natural separation of time 
scales allows orbit averaging over the particle 
trajectories (in the cases studied so far, this 
means averaging over many ion—cyclotron and 
axial—bounce periods in a magnetic mirror), and 
great reductions in the number of required par— 
tides have been achieved. Simulations have 
been performed with parameters that directly 
correspond to the Lawrence Livermore National 
Laboratory 2XIIB experiment. These simulations 
were able to simultaneously resolve the ion 
cyclotron time scale in the vacuum magnetic 
field w “ 2.8 x 10 s“l, and the ion-electron 
slowing down rate v«.i/e - 3 x 102 s ~l without 
artificial distortion. The new simulations 
required 500 to 1000 ions rather than the 
“"20,000 previously needed, had a correspondingly 
smaller memory requirement (were contained in 
core), and were able to run 10 to 100 more steps 
in the same amount of computer time (two to 
three hours on the CRAY 1). 

Extension of the orbit-averaged simulation model 
to include fluid electrons and provide a self- 
consistent implicit calculation of the ambipolar 
potential has been undertaken to simulate tandem 
mirror configurations. The inclusion 
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of electron effects requires that an even finer 
time scale than the ion cyclotron period be 
resolved, viz, the electron transit time in the 
mirror or mirror end-plug of the tandem. This, 
time scale is roughly a factor of ten faster 
than the ion gyration and will require that a 
realistic simulation span the additional spread 
in time scales. Because of the complexity of 
tandem mirrors with thermal barriers a signifi¬ 
cant increase in axial grid resolution is also 
required. Accommodating the first few azimuthal 
Fourier modes to incorporate quadrupole and 
elliptical flux-tube effects will further strain 
present capability. A 2 x 10^ word memory and 
a machine 2 to 4 times faster than a CRAY 1 will 
not suffice without continued artificial com¬ 
pression of time scales. 

Particle simulations of microinstabilities in 
mirror devices also strain the limits of capa¬ 
bilities of a CRAY 1 computer. Loss-cone 
simulations with a stretched one-dimensional 
code were performed with 0.5-20 x 10 particles, 
64-256 grid points, Ax~(0.05 - 0.2)pi and 
u ci At = 0.05 or 0.1. To accommodate the dispar¬ 
ate time scales of lower hybrid oscillations, 
ion cyclotron and ion bounce motion, the linear 
growth of microinstability, neutral beam charge- 
exchange, and ion drag, in the simulations, the 
rates of the slower processes have been artifi¬ 
cially accelerated (by as much as 10 2 ); and 
either w p £/tJ c 2 or mi/m e is at least a factor 
of ten smaller in the simulations than it is in 
mirror plasmas like 2XI1B and TMX end-plugs. 
Neverthelessj even witli somewhat artificial 
parameters and very simple models of the 
aspects of a neutral-beam driven mirror machine, 
the simulations of loss-cone modes give many 
results in agreement with experimental data and 
quasi-linear theory. The cost of these simula¬ 
tions scales directly with(m e /mi + w c j/wpi) - *' 
and the number of particles. Typical simula¬ 
tions require less than one hour on the CRAY 1 
with particle data stored on disks and input/ 
output overlapped, but a few of the simulations 
were as long as two to three CRAY CPU hours. 

More realistic microstability simulations of 
plasmas with parameters more closely approach¬ 
ing 2XIIB, TMX, TMX-U, and MFTF-B will be 
possible on an enhanced Class VI computer and 
easier still on a Class VII; and the code could 
be contained in core. Increasing the dimension- 
alitv and incorporating important electron and 
electromagnetic effect await a Class VII 
machine. 

The recent advent of the implicit particle 
codes, which allow large time step without com¬ 
promising microscopic physics, makes it possi¬ 
ble to run a particle code to examine slow 
phenomena such as drift waves within a reason¬ 
able computational time. This means that the 
particle simulation technique can describe an 
enriched and enlarged field of physics. Because 
implicit particle codes require storage of 
additional grid or particle data over that 
stored in conventional explicit codes, memory 
requirements are increased. Furthermore, the 
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desire to simulate experiments using fully re 
istic parameters in 2D or 3D .continues to den 
computer capabilities beyond Class VII. 

(d) Fokker-Planck Codes - In the SimulatTor 
magnetically confined plasmas where the ions a 
not Maxwellian and where a knowledge of the d 
tribution functions is important, kinetic 
equations must be solved. At number densitie 
and energies typical of mirror machines, end 
losses are due primarily to the scattering of 
charged particles into the loss cones in velc 
city space by classical Coulomb collisions. 
The kinetic equation describing this process 
the Boltzmann equation with Fokker-Planck 
collision terms. The use of this equation is 
not restricted to mirror systems. The heatin 
of plasmas by energetic neutral beams, the 
thermalization of alpha particles in DT. plasm 
the study of runaway electrons and ions In 
tokaraaks, and the performance of two-energy 
component fusion reactors are other examples 
where the solution of the Fokker-Planck equat 
is required. 


The problem is to solve a nonlinear, time- 
dependent partial differential equation for t 
distribution function of each charged species 
the plasma, as functions of six phase space 
variables (three spatial coordinates and thre 
velocity coordinates). Such an equation, eve 
for a single species, exceeds the capability 
any present computer, so several simpllfy^fe 
assumptions are required to treat the prcHH 

The most advanced state-of-the-art -time- 
dependent Fokker-Planck code assumes that the 
distribution functions depend on one spatial 
coordinate (radius r) and two velocity coordi 
nates (speed v and pitch angle 0). Moreover, 
the collision operator at each radius depends 
only on the distribution functions at that 
radius, so that a zero-spatial-dimensional, t 
velocity-space-dimensional Fokker-Planck solv 
may be utilized. 


Such a solver requires 11.5 us per meshpoint 
the CRAY 1 to compute the Fokker-Planck opera 
and time-advance the distribution function (u 
ing an alternating direction method) for one 
species. For a typical 1Q.I (v)by 81(0) mesh t: 
comes to 0.094 seconds per timestep per spec! 
Allowing for 10 radial meshpoints, two specie 
and 1000 time steps, the total amount of compu 
time is 31 minutes. 


The preceeding is not intended to give the id 
that any zero-spatial-dimensional, two-veloci 
space dimensional Fokker-Planck problem can b 
easily handled by the CRAY 1. Fusion efficie 
studies of the D-D fuel cycle require the sol 
tion of Fokker-Planck equations for five ioni 
distribution functions, including very hi 
energy protons. Using a state-of-the-art 
species Fokker-Planck code, the maximum a 
able mesh size on the CRAY 1 is approximately 
161(v) by 30(0), and the computer time per 
timestep (using a fully implicit method) is 
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12 sec. A typical problem requires anywhere 
from 1 to 4 hours. Moreover, there are many 
regions of parameter space which require two 
to five times -as many meshpoints. Some of these 
problems can be attacked on the CRAY-1S, at a 
premium cost, andl others will require an 
extended Class VI or a Class VII machine. 

There are many situations in which the charged 
particles execute regular orbits on a timescale 
much faster than their collision time. In such 
cases the Fokker-Planck equation need not be 
solved everywhere in space; instead bounce¬ 
averaging can be employed. Toward that end a 
two-velocity-space dimensional zero-banana- 
width Fokker—Planck solver has been developed. 

In its first approximation the ambipolar poten¬ 
tial in the bounce-direction is ignored, there¬ 
by simplifying the orbit equations. 

For a 101(v) by 81(0) mesh with 25 axial (z) 
positions (over which the Fokker-Planck co¬ 
efficients are averaged) 0.32 sec per timestep 
are required. The Fokker-Planck-related storage 
requirement is about 400,000 words. When placed 
in a radial transport code with 25 radial 
points, 8 sec per timestep will be "required, so 
that a typical 700 timestep problem will take 
93 minutes. 

Let it be emphasized that the above 93-minute 
problem is a gross simplification of what is 
needed. In mirror-like devices such as tandem 
mirror plugs and EBT, the axial electric field 
cannot be ignored. . Since the raidplane trans¬ 
formation (v, 0)-»(vq,0o) now depends on v, to 
use a similar algorithm requires, for a 101- 
point v-mesh, 101 times as much storage, which 
is clearly prohibitive. It will therefore be 
necessary to recompute the transformation arrays 
each timestep; this should result in a factor of 
at least 2 increase in computer time. 

More important, the zero—banana—width assumption 
is totally invalid for modeling neutral beam 
injection into a tokamak. Including a finite 
banana width will not only greatly increase the 
storage, but it will also result in a factor of 
at least 5 increase in computer time, so that a 
typical one-species 700 timestep problem on a 
101(v) by 81(0) by 25(r) mesh with 25(z) posi¬ 
tions will take, on a Class VII machine, 465 
minutes. 


cost of 1.5 x 10“3 seconds per time step per 
mesh point on the CRAY 1, and assuming th;at a 
calculation requires 200 time steps, the amount 
of CRAY 1 computer time required is about.10 
hours, generally an unacceptable amount of time 
for a single run. Incidentally, the total of 
storage required would be about 50% greater 
than the matrix size or about 3.4 x 10*> words. 
This could be accommodated only on an extended 
Class VI or on a Class VII machine. Assuming 
that these machines are respectively 4 and 10 
times faster than the CRAY 1 implies that this 
calculation would require 2.5 hours on the 
extended Class VI and 1 hour on the Class VII 
machine. Consequently the extended Class VI 
would be only marginally adequate both in terms 
of speed and storage for this calculation, 
whereas on a Class VII computer the problem 
would be tractable. 


In summary, as the fusion program has advanced 
rapidly in the last few years with the develop¬ 
ment of more sophisticated theory and experiment, 
computational requirements for accuracy and 
realism have increased to the point that Class 
VII capabilities and beyond are urgent. 

It is not possible to define a performance level 
that represents the ultimate capability for 
fusion studies. Each successive generation of 
supercomputers has been eagerly awaited by the” 
user community. Codes to exploit the new hard¬ 
ware capabilities are typically under develop¬ 
ment before the hardware is actually installed. 

It is safe to assert that the fusion computing 
community can effectively use the best perfor¬ 
mance that the supercomputer manufacturer’s are 
capable of providing for the foreseeable future. 
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I• INTRODUCTION 

the^enfr^ent^f 1 ? 82, th * D ^ p ^ tn,ent of De *«>se, National Science Foundation, 
organized a "Workshop\ ^ ^ at i° nal Aeronautics and Space Administration 
Thf Charter it ? ° n Lar ge-Scale Computing for Science and Engineering." 

vidinv lar^ fr ? WOrksh °P 15 t0 ^tennine needs and examine methods of pro- 
viding large-scale computing capabilities to science and engineerinv- also to 

of U ?L hC ° f ? oaputeT technology in the United State!. Further details 

Alamos Nat' S °? bC found in Appendix A. The Computing Division of Los 
Alamos National laboratory was invitod to participate and to prepare thisposi- 

cesearcfa 6 community^' U ° Uld P ” Vidt Rational Lp^biUty t/tll. 


A. What Los Alamos Can Provide. 


the wfr?H n ° nC ° f thC °° St powerful scientific computing facilities in 

lc,enrl d ' ° Ur mi f Slon 15 to P rov ide the computing resources required in 
blfd ? ffd research a " d large-scale numerical simulation. Thus we have assem¬ 
bled a wide variety of hardware, software, communication facilities, and ser¬ 
vices. In carrying out our mission, we seek to maximize first the productivity 

matew”™™ ”; ond J the Productivity of hardware. Currently, we have approxi- 
oately 3000 validated users within los Alamos County and another 500 use?s dis- 
tribute” throughout the United States. Although our users are engaged "a 

°f a PP llca ^ lons r angmg from document preparation to large-scale 
scientific simulation, they have the following basic needs in common: 


• suitable hardware/software, 

• convenient access, 

• mass storage, 

• a menu of output options, 

• support services, and 

• ease of use. 
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1. Suitable Hardware/Software. 
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The nucleus of our computing facility is a collection of computers designed 
primarily for scientific computation. , They are described in Table I. 


TABLE I 


COMPUTERS USED FOR SCIENTIFIC COMPUTATION 


uantitv 


Description 


eratine System 


Cray 1 


Interactive (CTSS) 


CDC-7600 


Interactive (LTSS) 


CDC-6600 


Interactive (NOS) 


CDC Cyber 73 


Interactive (NOS) 


DEC VAX 11/780 


Interactive (VMS and UNIX) 


The Cray Is and CDC 7600s are used for number crunching. The CDC-6600 and 
Cybers provide continuity with the past while supporting administrative comput¬ 
ing. The VAXs are incorporated into a distributed network called XNET. They 
are used to support experimental facilities controlling experiments, collecti^s 
and analyzing data, while at the same time providing a modern software-rich e^A 
vironmeat, for example, screen editing, virtual memory, Fortran 77. 


Note that an interactive operating system is provided on every computer. 
Experience shows that interactivity maximizes the productivity of users. 


2. Convenient Access. 

The typical user must frequently refer to notes, books, documentation, etc.; 
thus, the most convenient point of access is from the employee's office. We 
provide this through an integrated network exploiting our interactive operating 
systems. A functional diagram of the network is shown in Figure 1. Our net¬ 
work is partitioned into three security partitions: Secure, Administrative, 
and Open. For the purposes of this discussion, we need consider only the Open 
partition. That partition contains the following computers: 

• 3 CDC 7600s, 

• 1 CDC Cyber 73, and 

• 9 VAX 11/780S. 

A user in the Open partition can sign onto any of the above computers. 

Further, we would like to acquire a Cray 1 computer for the Open partition 
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3. Mass Storage. 

Supercomputers can consume and generate enormous amounts of data in a short 
time. Thus every large-scale computing facility needs a mass storage facility 
of considerable capacity. Most large-scale computing facilities in this coun¬ 
try recognize this need; however, few of them have mass storage facilities with 
sufficient efficiency and reliability to be quality systems. Our system is a 
quality sys’tem using IBM equipment and is shown in Figure 1 as the Common File 
System (CFS). This equipment has a large market among banks.and insurance com¬ 
panies and thus there is significant customer pressure to make the equipment 
work and keep it working. Currently, our system has an online capacity of 2.7 
trillion bits and unlimited offline capacity. To date, its availability aver¬ 
ages 98-99%. The file organization within it is tree structured much like 
UNIX. One of its novel features is automatic file migration. Storage within 
the system is hierarchical. The 3350 disks provide the first level of storage, 
the IBM 3850 cartridge store provides the second level and offline cartridges 
provide the third level. Recent performance data from this system are given in 
Table II. 


TABLE II 

RECENT CFS PERFORMANCE DATA 


Level 

of 

Storage 

% of Total Data 
Contained 
in this Level 

% of Requests 
Satisfied 

From this Level 

Average Respons 
Time 

From this Leve 

Disk 

1 

82 

10 seconds 

3850 

17 

17 

1 minute 

Offline 

82 

1 

5 minutes 


We believe this is one of the finest mass storage systems in the world. 
It is accessible from all of our computers and utilization of it would 
be included in our provision of large-scale computing facilities. 


4. Output Options 

In code development and exploratory computations, the scientist often needs a 
variety of output options. For example, during code development and problem 
checkout, graphical display at the terminal can be extremely valuable. During 
the course of a parameter study, one may wish to put copies of the source and 
numerical results on microfiche for efficient archival storage. If one is 
dealing with a time-dependent problem that consumes several hours of supercom¬ 
puter time, then one may wish to produce a movie to show time dependencies. 
Finally, in code development, there is the time-tested axiom "when in doubt, 
count out." All of these options are available in the Los Alamos Computing 
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Facility through the equipment listed in Table III (shown in Figure l.as PAGES, 
Print and Graphical Express Station). 


TABLE III 

OUTPUT OPTIONS AND EQUIPMENT 


_ Output Potion _ 

8 1/2- x 11-in paper (double sided) 
11-in roll (electrostatic) 

36-in roll (electrostatic) 

36-in vellum (electrostatic) 

16-mm color film 

35-mm color and black and white film 
105-mm microfiche 


_ Equipment _ 

2 Xerox 9700s 
Versatec 
Versatec 
Versatec 

FR80 film recorders 
FR80 film recorders 
FR80 film recorders 


PAGES is accessible online from all of our computers and utilization of it 
c ould be included in our provision of large-scale computing. 


5. Support Services 

Requisite.support services include 

• operations, 

• documentation, 

• education, 

• consulting, 

• accounting, and 

• research. 


We are particularly proud of the efficiency in our operations. Excepting the 
VA_\ 2 1 /780s, all of our computers, the Common File System, and the output 
equipment in PAGES are operated 2U hours a day, 363 days a year by a total of 
76 people. As evidenced by CFS and PAGES, part of our objective is to automate 
where possible. Thus, our Facility for Operations Control and Utilization 
Statistics (FOCUS in Figure 1) is a recently added node to our network from 
which we load level production jobs across the supercomputers. Because of au¬ 
tomation, we anticipate no growth in our operation staff in the next few years 
despite plans to significantly increase our total computing capacity. 
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Documentation is the Achilles heel of computing. The Los Alamos Computing m 
Division is organized into eight working groups. Because of the breadth of o^| 
network and unique facilities such as the Common File System, one of those 
groups has responsibility for developing and maintaining documentation. This 
is a measure of the importance we attribute to documentation. Much of our doc¬ 
umentation is available online and our goal is to put all of it online in the 
near future. 


Education is helpful to new users and we already make considerable use of video 
cassettes and computer-aided instruction. Should we offer service to a nation¬ 
al computing community of users, these would probably be the primary media for 
education. 

Consultation is required when users encounter difficult bugs or when they seek 
information about how to accomplish sophisticated tasks. Today we have a staff 
of approximately 10 consultants; this staff would have to be increased should 
we offer service to a larger community. Since the introduction of interactive 
systems, the consultants' primary mode of communications with users has been by 
telephone and it would likewise be the primary mode should we provide service 
to a national community■ 

The objective of our accounting system is to charge equitably for all resources 
consumed. We charge for CPU utilization, memory utilization, storage of infor¬ 
mation in the Common File System, number of pages printed, frames of film gen¬ 
erated, etc. A summary of projected charges for FY 83 are attached in Appendijt 

• 

Our research divides into three areas: 

• parallel processing, 

• person/machine interface, and 

• modeling support. 

The next generation of supercomputers will likely incorporate parallel process¬ 
ing and will be available by 1985. Preparation for them is driving much of pur 
research. Because of the processing power soon to be available in desktop per¬ 
sonal workstations, we believe that the person/machine interface will undergo 
radical changes in this decade. This will generate significant new require¬ 
ments in networking and perhaps change the way we do scientific computing. 

Good computer modeling includes development and analysis of mathematical 
models, implementation of mathematical models into software, and validation of 
both. These-are important areas of research for any scientific computing or¬ 
ganization. 


6. Ease of Use 
Ease of use includes 

• specialized supercomputer software, 

• software-rich systems, and 

• common software across the network. 
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Our speei^lizeid supercomputer software includes symbolic debugging tools and 

machine-language subroutines for vector operations. a!so op- 
con PJ lers provided. Software richness is achieved by a variety of 
computing systems, a menu of programming languages, and a variety of vendor- 

sys?ems d throieh a e 10nS CoB “°°ality is provided across all computing 

systems through common libraries of graphics and mathematical software and 

PAGES' COm “° D utlllties for communication with the Common File System and with 


II. 


NATIONWIDE ACCESS TO THE LOS ALAMOS CENTRAL COMPUTING FACILITY 


trai r ro^ idin f a -T 3 f iety ^°! communication services into the Los Alamos Cen- 
, .®P in 8 Facility. Today, we have telephone-dialup access at 300/1200 

asynchronous terminal service. We also have telephone access 
at 1200/2400 bit/s for 200 UT user stations to access the NOS systems. There 
several telephone dedicated leased lines that provide service Leased line 

Pr ° V n deS 9 ?°° /56k bit/$ - In addition > the'telephone^any c n pro¬ 
vide Direct Digital Service (DDS) at 9600/56k bit/s. P 

^ss^M ri rh ntal h S ? f I :Ware SySte ° “ runnin 8 in one V AX at Los Alamos that is ac- 
- sible through Telenet. That service provides 300/1200-bit/s service We 

the eCT«taUou'tLt Tel "“ * ccess into th ' Computing Facility with 

the “®.‘ raSS thr0U811 T ' len “ UU1 bt 

Jut the t r!ir P ^ n _ increasin | communications capacity is to acquire equipment to 
put the Central Computing Facility onto ARPANET. Purchase orders and other ad¬ 
ministrative matters are complete, and we are awaiting the deliver^ of UeCSO 
processor from Bolt, Beranek & Newman to complete this task. The^RPANET serv¬ 
ice should be available at Los Alamos by the beginning of 1983. 

Los Aiamos is also part of the magnetic fusion energy network. W e are current- 
Mt! d ^ AmeriCan Satellite w *tfa dual 56k channels on the magnetic fusion 

The ^rtment of Energy has an ambitious project to acquire and place in serv¬ 
ice wideband satellite communications nationwide. This project is called 

Th '.j°“ nt ° f 0PM0DEi i* t" provide uideLSd doc, 
service, voice, and full-motion video-conferencing capability. The first serv- 

«“ °° OPMODEt for DOE sites will become svsilabfe about October 1983 We ex- 
pec that there Will be local telephone access into some of the 0PM0DEL sat- 

1984-8 5 8 timeframe^ dOQS * ^ ^ iS 3 te " deVel °^’ P»bably in the 

U*H b oi ieV r tha ? bbe integrated network that has been developed and is in place 
^operational at Los Alamos is among the finest in the world. The network is 
jtable, it has a high reliability, and we can provide service as TTnmr. H Qf 
course, there is lead time between the time of request for service and the time 

that we can provide it. Today that lead time varies between three and six 
"ninths. ' 
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HI. WILL THESE FACILITIES SERVE EDUCATIONAL NEEDS? ^ 

The Los Alamos Computing Facility provides supercomputers, mass storage facili¬ 
ties, a menu of output options, and interactive access plus requisite support 
services. A recent user satisfaction survey by a professional firm shows that 
our computing facility is meeting the needs of the scientific user We believe 
that it will also meet the needs of educational institutions. 


IV. HOW SHOULD THESE FACILITIES BE MANAGED AND FINANCED? 

These facilities should be managed by the Computing Division of the Los Alamos 
National Laboratory with an advisory panel of at most six people representing 
the interests of the research community. Financing should be by recharge for 
resources consumed and services rendered. 


V. PLANNING AND POLICY STRUCTURES 


The Los Alamos Computing Division produces annually a two-year operational 
plan. The operational plan is driven by programmatic requirements, user re¬ 
quirements, and technology trends. Copies of the FY 82—83 Two-Year Plan are 
available on request from the authors and the reader is encouraged to consult 
Chapter 2 of it for discussion of the planning process. 


Policy would have to be negotiated between Laboratory management and an adm 
istrative body representing the.research community. 
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APPENDIX B 

RATES FOR CCF SERVICES 


CCF Service 
Computing Services 

CTSS (Recorded Hour) 

LTSS (Recorded Hour) 

NOS (Recorded Hour) 

Tape Mount (Each) 

ICN Services 

300-bit/s Port/Month 
1200-bit/s Port/Month 
9600-bit/s Port/Month 
150-kbit/s Port/Month 
Intelligent Workstation/Month 
200 UT Port/Month 
CBT Port/Month 
56-kbit/s XNET Port/Month 
256-kbit./s XNET Port/Month 
Connect Hour (300 bit/s) 
Connect Hour (1200 bit/s) 
Connect Hour (9600 bit/s) 
Connect Hour (150 kbit/s) 
Connect Hour (200 UT) 

Network Transmission (megaword) 


FY 1983 


422.50 
295.00 

407.50 
3.75 


90.00 

135.00 

180.00 

270.00 

270.00 

720.00 

1080.00 

1620.00 

3240.00 

2.70 

4.15 

5.50 

8.25 

7.20 

3.75 


CFS Service s 

Online Access (Each) 

Offline Access (Each) 

Online Storage (megaword Month) 
Offline Storage (megaword Month) 

PAGES Services 


Printed Output (Pages) 
Megabytes Processed 
105-mm Fiche (Each) 
35-mo Film (Frames) 
16-mo Film (Frames) 
Plotter (Sheets) 

VMS Services 


Monthly Fee 


0.32 

3.75 

13.75 

2.62 


0.20 

4.05 

4.05 

0.22 

0.11 

0.33 


262.00 
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Facility through the equipment listed in Table III (shown in Figure 1 as PAGES, 
Print and Graphical Express Station). A 


TABLE III 

OUTPUT OPTIONS AND EQUIPMENT 


_ Output Option _ 

8 1/2- x 11-in paper (double sided) 
11-in roll (electrostatic) 

36-in roll (electrostatic) 

36-in vellum (electrostatic) 

16-mm color film 

35-mm color and black and white film 
105-mm microfiche 


_ Equipment _ 

2 Xerox 9700s 
Versatec 
Versatec 
Versatec 

FR80 film recorders 
FR80 film recorders 
FR80 film recorders 


PAGES is accessible online from all of our computers and utilization of it 
•>uld be included in our provision of large-scale computing. 


5. Support Services 

Requisite support services include 

• operations, 

• documentation, 

• education, 

• consulting, 

• accounting, and 

• research. 


We are particularly proud of the efficiency in our operations. Excepting the 
VAX ll/780s, all of our computers, the Common File System, and the output 
equipment in PAGES are operated 2U hours a day, 363 days a year by a total of 
76 people. As evidenced by CFS and PAGES, part of our objective is to automate 
where possible. Thus, our Facility for Operations Control and Utilization 
Statistics (FOCUS in Figure 1) is a recently added node to our network from 
which we load level production jobs across the supercomputers. Because of au¬ 
tomation, we anticipate no growth in our operation staff in the next few years 
despite plans to significantly increase our total computing capacity. 
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Supercomputers and the Equations of State 
of a Classical One Component Plasma 


Bill Buzbee 


Following the pioneering work of Brush, Hansen, and Teller(l), scientists 
have been attempting to compute the equation-of-state (EOS) of a classical 
one-component plasma (OCP). OCP is an idealized system of ions iiraiersed in 
a uniform sea of electrons such that the whole system is electrically 
neutral. Though OCP is a "simple material", if its EOS could be computed, 
a framework would be available from which to address the EOS of more 
coroplicated substances. Prior to the advent of the Cray-1, scientists had 
not been able to compute the EOS of OCP throughout the energy interval of 
interest. 

The EOS can be calculated with a Monte Carlo technique. The density and 
temperature in the system are fixed, and the system energy is computed. 

Then a particle (ion) is selected at random and subjected to a small, 
random perturbation (movement). The system energy is recalculated, and the 
move is accepted if the energy decreases. If the energy increases, the 
move is accepted with a probability described by the Boltzmann 
distribution. Now another particle is chosen at random, ... ad infinitum. 
Once the energy stabilizes, a point of the EOS is in hand. New values of 
density and temperature are chosen, etc. 

Calculation of the system energy requires evaluation of an interparticle 
potential. The potential is not available in closed (finite) form, so an 
approximation must be made and the approximation must be computationally 
affordable on the computer. With the availability of the Cray-1, 
scientists (2) developed an improved (more complex) approximation to the 
interparticle potential. By using very efficient vector algorithms and 
software developed under the Applied Mathematics Program of DOE s Office of 
Basic Energy Science, it was possible for the Cray-1 to execute the new 
calculation at about 90 million floating point operations per second. Even, 
so, some seven hours are required to carry out the EOS computation. Most 
important, for the first time, the EOS for a OCP was accurately calculated 
throughout the interval of interest. Further, the melting temperature was 
found to be ^20% lower than the widely accepted previous estimates. This 
advance is a direct result of the high level of performance available from 
the Cray-1. 

This example also illustrates an important point about algorithms for 
vector computers. Throughout the era of electronic computation, our 
research has sought to develop algorithms that have a minimum number of 
arithmetic operations. Indeed, this is the correct approach for scalar 
computers, but it is not necessarily the correct approach for vector 
computers. Evaluation of the interparticle potential approximation 
requires evaluation of the complimentary error function. The complimentary 
error function contains an integral over a semi-infinite interval. Thus it 
must be approximated for efficient computation on a computer. On 
sequential (scalar) computers, scientists use a very simple and sensible 
approximation —, namely, they tabulate the function and evaluate it via 
table lookup and interpolation. Unfortunately, table lookup is inefficient 
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on a vector computer. A better approach on a vector machine is to use a 
polynomial approximation thereby eliminating table lookup. This approach • 
involves many more arithmetic operations than the tabular approach, but 
they are fully vectorizable and can be executed at a very high rate. So in 
this case, the best algorithm for a vector computer is quite different from 
that for. a conventional computer. 

This example of computing the-equation-of-state for a classical one- 
component plasma illustrates how increases in supercomputer performance 
make possible advances in fundamental knowledge. It also shows how changes 
in computer architecture can impact computer simulations and the payoff of 
research to find suitable algorithms for them. 


1. S. 6. Brush, H. L. 
(1966). 

2. W. L. Slattery, G. 
2087 (1980). 


Sahlin, and E. Teller, J. Chem. Phys. 45, 2102 
D. Doolan, and H. E. DeWitt, Physical Rev A 21 (#6), 
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Aerospace Structural Design and Supercomputers 
Richard H. Gallagher 

The products of the aerospace'industry — commercial and military 
aircraft and aerospace vehicles — represent extremely challenging 
structural design problems. They must be of the lightest possible weight* 
perform safely, and should strive for the lowest cost of manufacture. They 
are of complicated form and in each succeeding decade they are planned to 
operate in more severe environments and to incorporate new materials and 
design concepts. Because these challenges have been met up until now, the 
United States has been and continues to be foremost in the aerospace 
industry and that industry is a major factor in our economy and balance of 
paymentsi 

rrwi tl-e onset of electronic computation, the aerospace design com- 
T.v.clry nas (;..p 1 '."/eo the largest computers available to commence! enterprise 
,;t ,%'iy gi von time. At. first, in the early 1950’s, the ex sting computers 
„ero app^ed to static analysis in terms of a few unknowns. The scale of 
analyses;advanced in tandem with the development of Class II - V computers, 
r.ven today the unknowns of the real structure outdistance the capacities of 
class V machines. If design optimization, computer graphics, and dynamics 
phenomena are treated, then there must; be tradeoffs such that the computer 
descriptjio'i falls substantially short of the real structure. Class VI - 

machines, which are inwninent participants in the aerospace structural design 

» . 

process, will make inroads into this discrepancy but will not resolve it 
fully. ; 

The above remarks refer to structural design in isolation. Major 
strides are being made in design for aerodynamics, especially in the 
definition of shaues for drag reduction. As noted in Section III of the 
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report, the computations resources that are needed for more effective 
aerodynamic design are 100 times greater that those of Class VI machines. 
Inevitably, aeroelastic interaction must be taken into account, wherein 
the structural and aerodynamic behaviors are dealt with in an integrated 
computational exercise. The computational needs will tax even supercomputers. 

The requirements increased fuel efficiency, reduction of material 
weight, greater range, and reduced manufacturing costs are quite evident. 

The international aerospace industry is aware that these goals can be 
achieved through realizable, expanded computer power. The U.S. segment of 
that industry is a prime candidate for loss of its pre-eminent position if 

the computational tools available to it do not move ahead with the sate-of- 
the art. 
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Supercomputers and Engineering Research - Brice Carnahan, University of Micf 

To date. Class VI machines have not been available to a broad 
spectrum of engineering researchers, particularly academic 
researchers. Despite this, these machines have already had a flB 
significant impact on research in several engineering disciplines^^ 
The problems tackled so far are characterired less by the particular 
discipline than by the mathematical formulations involved. Broadly 
speaking, the following appear to be the most important: 


(1) Solution of linear apd nonlinear PDE describing transient 
(unsteady state, dynamic) behavior using finite difference methods. 
Typically, the solutions involve of the order of a million nodes 
(say 100 per space dimension). Major applications are the 
solution of problems in fluid mechanics (e.g., Navier-Stokes 
equations) in aeronautical engineering, heat transfer in 
mechanical engineering, and flow through porous media (reservoir 
simulation) in chemical and petroleum engineering. 

(2) Solution of PDE describing static (steady-state) behavior 
using finite element methods. Typical problems involve of the 
order of 10,000 element nodes and direct solution methods 

are used to solve the nodal equations. Principal applications 
have been in the structures area (particularly for aerospace 
structures). Some fluid mechanics and heat transfer problems 
have also been solved using this approach. 


(3) Monte-Carlo methods involving of the order of 1 
particles. A major application involves solution of 
transport equation in nuclear reactor shielding anal 


000 

the radiation 
yses. 


(4) Real time image processing. Extremely fast filtering, 
interpolation, and back-projection calculations are required 
for analysis of high-resolution video signals, such as are 
generated in tomographic equipment. 


There are problems in each of these areas that could profitably 
use machines 100 to 1000 times faster that the current Class 
VI machines. For example, the finite element method could be 
used to solve transient problems involving perhaps 100,000 
element nodes. This would require the solution of thousands 
of simultaneous ordinary differential equations in addition to 
the direct solution of large numbers of (typically linear) 
equations. Faster, larger machines would allow the use of 
much 1arger sample sices in Monte—Carlo simulations with 
attendant improvements in the accuracy of problem statistics. 
Many of the image processing problems, particularly those 
associated with real-time nondestructuve testing, will be 
difficult to solve without much faster machines. 


Probably the greatest impact that the supercomputer of tomorrow 
will have on engineering will be the solution of systems problems 
that simply cannot be tackled at all with current equipment. 

For example, in aerospace engineering, the fluid mechanical, j 

structural, and control systems analyses are treated in isolation. 
In the future, models of the integrated systems will be studied, 
and will undoubtedly lead to far better, more efficient aero¬ 
space vehicles. 
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Similarly, in chemical engineering, it should 
be possible to study the dynamic behavior of entire chemical 
processing complexes (currently almost all chemical systems 
designs are based on steady-state models and any associated 
control systems are designed using much simplified dynamic 
models of plant subsystems) requiring the solution of hundreds of 
thousands of simultaneous algebraic and differential equations. 
For the first time, it should be possible to use nonlinear 
programming algorithms to optimize the design and operation of 
such nonlinear systems, which incorporate a significant number of 
design variables. Automatic process synthesis (given only 
specifications for raw materials and desired products) 
algorithms may well be feasible, with sufficient computing 
power. 

Undoutedly, similar examples involving the design, simulation, 
optimization, and even operation of complex systems can be 
found in virtually any engineering discipline. 
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SOME EXAMPLES OF SUPER COMPUTER PROJECTS 


L. L. SMARR 


UNIVERSITY OF ILLINOIS 


* High Resolution Gas Dynamics - The large memory and high speed of super¬ 
computers allows one to simulate dynamic gas flows with high resolution. 
Astrophysical examples are the flows that occur in quasars and radio gal¬ 
axies. These can involve the central powerhouse of active galactic nuclei 
- nonspherical accretion onto supermassive black holes [J. Hawley and L. 
Smarr (Univ. of Ill.) with J. R. Wilson (LLL) - calculations performed on 
CRAY-1 at LLLl. Or they can involve the formation, propagation, and sta¬ 
bility of supersonic and relativistic jets in these objects [calculations 
by MUNAC (Munich Numerical Astrophysics Coalition) - M. Norman (MPI), K.- 
H. Winkler (MPI) and L. Smarr (Univ. of 111.) on the MPI CRAY-1 and by J. 

R. Wilson (LLL) on the LLL 7600]. The use of large numbers of grid points 
(- 5 x 10^) allow theoretical resolution of the flows comparable to the^P 
observational resolution of the Very Large Array of radio telescopes or 
the Einstein orbiting x-ray telescope. 


* Numerical Relativity - Here one solves the fully time-dependent Einstein 
equations of general relativity to study the strongly curved spacetime 
near b^ack holes or in the early universe. Again, many grid points 
(> 10^*) are needed to cover both regions near the strong gravitational 
field as well as far away. Recent projects computed include formation of 
nonspherical black holes and neutron stars, generation of gravitational 
waves, and distribution of the dark matter in the universe [J. Centrella 
(Univ. of 111.), L. Smarr (Univ. of Ill.), P. Dykema (LLL), C. Evans 
(Univ. of Texas), and J. R. Wilson (LLL) - calculations on LLL CRAY-1]. 

d 

Detailed calculations of these nonlinear processes are needed by the ex-^ 
perimentalists designing gravitational wave detectors. 
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* ■ —_y ln g Quantum Ch r omodynamic s - Exciting progress is being made in the 
program, started by Kenneth Wilson and Alexander Polyakov, to gain insight 
into the nature of the strong interactions by applying statistical methods to 
a formulation of quantum chromodynamics on a discrete spacetime lattice. For 
example, a recent breakthrough investigating the range of forces responsible 
for breaking chiral symmetry using a lattice of fermions was carried out nu¬ 
merically at Los Alamos using CDC 7600's [J. Kogut, M. Stone, and H.W. Wyld 
Call of Univ. of Ill.), J. Shigeraitsu (Brown), S.H. Shenker (U of CA) and D.K. 
Sinclair (Stanford]. The work must be limited to spacetime lattices of 5 x 5 
x 5 x 10 points because of memory and spacetime speed limitations. Clearly a 
supercomputer would enable much more detailed work to be done immediately. 

S evere Storms, Tornados, and Downbursts - The basic characteristic of 
storms which spawn tornados can now be simulated using CRAY-1 level computers 
le.g. K. Wilhelmson (Univ. of Ill.) and others - calculations on NCAR CRAY). 
The development, l^dimensional structure, and movement of these storms can be 
modeled and compared with observed wind fields from Doppler radar. With 
enough supercomputer time, these studies can be extended to understand the de¬ 
tailed formation of tornados, hazardous downbursts, hail growth, and damaging 
surface winds associated with convective storms. 


* iH- ramolecular Dynamics Calculations - Realistic calculations of intra¬ 
molecular energy flow in even moderate size molecules (e.g. five to eight 
atoms) requires large computers. Using small computers (l.e. CYBER 175 class) 
many groups have done classical calculations comprising a few trajectories at 
unrealistically high energies. Some work was done in the mid 1970's on the 
largest available computer (Illiac IV by J.D. McDonald (Univ. of Ill.)]. ^ 
remains the only work with a statistically significant number of trajectories. 
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What Is really needed are quantum calculations on Intramolecular energy tra 
fer. These calculations are completely hopeless without the latest generation 
supercomputers. Even on these computers they count as large scale work. 


Wilson 


EXAMPLES AND NEEDS OF SUPERCOMPUTERS 
K. Wilson 

Cornell University 

Thermodynamic Properties of Fluids 

A research group consisting of Keith Gubbins, William Street, and 
S. Thompson, in the Chemical Engineering department at Cornell, use both 
a shared Floating Point Systems Array Processor and their own mini¬ 
computer for simulations of liquids at the molecular level. At the 
present time these simulations can only supplement information obtained 
from analytic theory and experiment, due to the limited sample sizes 
(numbers of molecules) that can be studied in a practical simulation. 
Availability of computers thousands of times more powerful than current 
supercomputers would encourage this group to give heavy emphasis to the 
determination of thermodynamic properties of fluids directly from 
molecular potentials via computer simulation. Such a capability would 
be of enormous significance for many engineering applications. 

Theory of Nuclear Forces 

One of the most spectacular developments of basic theoretical 
research in the 1960's was the discovery of gauge theories providing a 
unified description of nuclear interactions, weak interactions and 
electromagnetic interactions. The Glashow-Weinberg-Salam gauge theories 
of electromagnetic and weak interactions are well understood theoretic¬ 
ally and have received impressive experimental confirmation. In con¬ 
trast, all that can be said so far about the gauge theory of nuclear 
(strong) interactions—called Quantum Chomodynamics—is that it is 
not obviously wrong. (It is however virtually the only theory of 
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strong interactions with this property.) The reason for the poorly 
understood state of Quantum Chromodynamics is theoretical: no analytic 
techniques are known for solving this theory, not even for approximately 
solving it. A major international effort has been underway for several 
years to try to solve quantum chromodynamics numerically, to obtain its 
predictions for fundamental quantities such as the masses and magnetic 
moments of the proton and neutron. Theoretical groups in the U.S. at 
M.I.T., Cornell, Brookhaven National Laboratory, Princeton, Columbia, 
Cal. Tech., the Institute for Theoretical Physics at Santa Barbara, 
Stanford Linear Accelerator Center, University of Chicago, University of 
Illinois, University of Indiana, etc. have engaged in this effort; 
mostly using borrowed computer time on minicomputers and array pro¬ 
cessors] Present day computers have been grossly inadequate for a 
complete, reliable computation; the current results involve simplifying 
assumptions and are unreliable even after the simplifications. 

As a result of computations done so far, it is clear that a com¬ 
plete, creditable computation will stretch the very limits of future 
computing technology. Such a computation will probably require thou¬ 
sands or millions of specially designed, very high speed processors 
embedded in a highly parallel network for data interchange. (The basic 
computational problem here is that sparse matrices of size roughly 
10 6 xl0 6 must be inverted at every step of a Monte Carlo numerical inte¬ 
gration; the Monte Carlo calculation involves of order 10 6 integration 
variables; the answer must give high accuracy for some very small long 
range correlation functions.) While many of the quantities to be 
computed are already known experimentally, the success of the calculati 
is of fundamental importance to the understanding of nuclear forces 
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and will also lay the basis for new, nonperturbative studies of particle 
interactions at super high energies. More generally the technology 
established for this ^calculation is likely to lead to equally important 
computations in many other areas of basic and applied scientific research 

The Renormalization Group and Statistical Mechanics 

A major breakthrough in theoretical research in the last two 
decades was the discovery of a new framework for understanding a whole 
class of previously unsolved problems. 2 The problems addressed by 
renormalization group techniques includes critical phenomena (the onset 
of phase, transitions), quantum field theory, polymer problems, various 
problems in solid state physics, and turbulence in classical fluid flow. 
Analytic expansion methods were found to make realistic renormalization 
group analyses in special cases of some of the above problems—for 
example, an expansion about four space dimensions for critical pheno¬ 
mena. Unfortunately only a minute fraction of the problems addressed by 
the renormalization group framework can be solved with current analytic 
renormalization group techniques. For example, relatively little progress 
has been made in its application to turbulence. Attention is now turning 
to the use of computer simulation to support the renormalization group 
approach: the "Monte Carlo Renormalization Group method" of Swendsen and 
Wilson has provided accurate numerical results for some models of 
specific critical phenomena on surfaces that were previously intract¬ 
able. 3 However, full flowering of the renormalization group approach 
requires vastly greater computing power than is available today. Even 
for the prototype system for all renormalization group studies—the 
three dimensional Ising model of a ferromagnetic transition—current 
Renormalization group research is blocked by inadequate computing power. 
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Wilson and Swendsen have already spent hundreds of hours of computing 
time on an array processor in an effort to study the numerical trun¬ 
cation errors in the Monte Carlo Renormalization Group technique. Un¬ 
fortunately, this effort has failed so far; the calculations to date 
show that a thousand or even million fold increase in computing power 
will be required to identify sources of error and make sure each error 
decreases at the expected rate as the truncations are removed. This 
effort is critical to establishing the credibility of numerical Renor¬ 
malization Group methods and encouraging its use in many applications. 

The Ising model is best handled through special purpose VLSI chips. 

An example of such a chip, with spectacular performance, was partially 

4 

designed at the Institute for Theoretical Physics at Santa Barbara ^ 

using the Cal. Tech, chip design system. To make full use of such a chip^ 
requires thousands of chips be operated in parallel with high speed data 
paths linking the chips, and parallel general purpose computers to carry 
out supporting computations. It has been impossible under present funding 
patterns to reproduce the chip or set up this support framework; as a 
result the Santa Barbara chip has been more of a curiosity than a 
productive scientific instrument. Experience with a network of Ising 
chips would lay the groundwork for a far more multi-purpose, highly 
parallel, system needed for the nuclear force calculations that could 
handle a large range of statistical mechanical simulations, predicting 
the properties of bulk matter from basic interactions of molecular con¬ 
stituents. In constrast, some two dimensional statistical mechanical 
problems are already feasible; researchers at M.I.T. and the University . 
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of Georgia have carried out Monte Carlo Renormalization Group 
computations on two dimensional statistical mechanical models 
(in collaboration with Swenasen)^ with considerable success. 


'• See ’ e *9* H * Hamber, Brookhaven preprint (1982) for a recent 
1 1st of references. 

See * e *9‘ K - Wllson in Scientific American, Vol. 241, No. 2, p.158. 

3. See D. P. Landau and R. H. Swendsen, Phys. Rev. Lett. 46, 1437 
(1981). 

4. R. B. Pearson, J. L. Richardson, and D. L. Toussaint, Santa 
Barbara ITP report ITP-81-139 (unpublished). 

5. See Ref. 3 and R. H. Swendsen, P. Andelman, and A. N. Berker, 

Phys. Rev. B24, 6732 (1981). 
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Role and Needs for Large Scale Computing for Theoretical 

Kenneth 6. Wilson 
Cornell University 


Phvsici 


1. Statement of Need for Theoretical Physics as a Whole 

At the present time the principal trend in theoretical physics is 
the broadening of the scope (in practice) of theoretical ideas by the 
inclusion of ever more complex problems within the theorists' purview. 

This trend is taking place uniformly throughout every subfield of theory. 
This broadening is ^taking place through re-examination of old problems 
(such as turbulence), the development of new areas (such as supersymmetry' 
or grand unified theories of elementary particle physics) or the recog¬ 
nition of' fundamental questions in applied areas such as metallurgy. 

The current trend is making possible for greater contact between funda¬ 
mental theory and complex practical applications than ever before, and 
this is reflected in a new interest in hiring of theorists by industry, 
for example at Schlumberger-Doll Research and at Exxon. 

In consequence of the move towards complexity, theorists have outrun 
the capabilities of traditional analytic theoretical methods. Compli¬ 
cations dealt with today include multiple degrees of freedom, irregular 
geometries, multiple length scales, and random irregularities, any¬ 
one of which can defeat analytic techniques. Modern theory has conceptual, 
qualitative methods to treat complexity: the "renormalization group" is 
one very general framework that has emerged over the last thirty years 
with an enormous sweep. However, the qualitative concepts such as the 
renormalization group provides must be backed up by more quantitative 
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results. In many cases, numerical simulations are the only approach 
available for providing quantitative analysis. 

Unfortunately, the problems which are too complicated for analytic 
analysis can place very severe demands on a numerical approach. For 
example, each new degree of freedom that is included in a numerical 
treatment can easily cause a factor of 100 or 1000 increase in computing 
requirements (both for cycles and memory); a standard example is the 
change from a nonrotating, spherically symmetric star to a rotating star 
where the angle to the rotation axis is an extra degree of freedom. 

Problems with randomness, thermal fluctuations or quantum fluctuations 
typically have thousands or millions of degrees of freedom, which can 
only be treated by statistical Monte Carlo or statistical dynamics methods. 
Computing time needed for these problems increases as the square of the 
accuracy desired; the isolation of small errors and establishment of 
numerical reliability for these statistical calculations demands very high 
accuracy and hence very painful computing requirements. 

The building of a simulation capability for a complex problem can 
take many years of effort. In early stages there comes the pursuit of 
various algorithms to achieve both convergence and efficiency, along with 
supporting studies to make sure all necessary physical principles have 
been correctly taken into account in the numerical procedures. At later 
stages come the study of errors; in a complex situation there can be 
many sources of error with lots of cancellations. In this case naive 
estimates based for example on two calculations with modestly different 
levels of truncation can be totally misleading and years of lengthy calcu¬ 
lations and careful study -ay be required before the numerical simulation 
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can be performed with a reliable error estimate. The computing resourced 
needed for all stages of a simulating effort usually vastly outruns 
(by a factor of 100 to 1,000,000 or more) the computing time needed for 
a single production run once a reliable production code is established. 

A crucial practical aspect of large scale simulation is that when 
the computing demands of a simulation increase, the increase is not by 
a factor of 2 or less: the increase is usually a factor of 10 or 100 
or more. For example, simulations often involve a multidimensional grid 
approximating a continuum; to achieve a better approximation a minimal 
improvement involves doubling the number of grid points along each dimen¬ 
sion. In a three dimensional grid this means a factor of eight increase 
in the number of grid points overall. Typically, further complications 
develop so that the increased computing demand is considerably more thah^ 
a factor of eight. ^ 

In conclusion, it is normal and reasonable that the computing demands 
for scientific simulation are prodigious, even by comparison with today's 
most powerful supercomputers. In fact, the whole area of scientific 
simulation in support of modern theoretical physics has barely been 
scratched; providing full access to theorists on the best of today's 
computers, can only be viewed as the beginning of many further stages of 
providing new levels of computing capability in support of ever more 
complex simulations. The complexity in simulation that can be handled 
numerically must keep pace with the growth in complexity that theorists 
are able to handle conceptually. 

It is of major economic importance to the U.S. that the current 
broadening trend in theoretical physics be encouraged, and in particular. 
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tnat the computing support problem be rectified. The high technology 
economy itself is facing much more complex problems than ever before. 

OH used to come out of gushers; now it must be coaxed out and ir. future 
much more recalcitrant raw materials will replace oil. This fact under¬ 
lies the new interest in hiring of theorists by the oil industry. Simple 
industrial materials like steel are being replaced by ever wider ranges. 


of composites; mastering their properties and designing new ones quickly 
(to stay abreast of competition from abroad) can be helped by theoretical 
studies, if these studies can be comprehensive and accurate. Simu¬ 
lations of products and processes throughout industry can be cheaper and 
faster than experimentation and prototyping and provide more opportunity 
for optimisation or for meeting conflicting requirements. In all these 
applications new problems requiring the attention of theorists will arise; 
theory will grow and grow in importance as the capabilities of theory 
become more powerful. It would be folly to block the expanding capability 
of theory by continued denial of access to computing. 


*• Practi cal Problems of Computing Support 

There are three issues which I think are not adequately dealt with 
in the body of the workshop report. 

1) Obtainins data that establish in detail the scientific importance 
and quality of scientific simulations. 

2) Providing adequate capacity despite rapidly growing demands for 
computing. 

3) Management and Manpower issues. 

1) None of the committees I have served on, including this workshop, 
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have solved the problem of documenting the importance of computing 
support for theorists. Members bring lists of needs within their 
fields, but these lists seldom provide any overview of the import¬ 
ance of the needs. There must be a more ongoing effort to collect 
data establishing the scientific importance of theorists' computing 
needs, especially given the skepticism of many eminent scientists 
towards large scale computing. This should be done by collecting 
reports on computing needs on a continuing basis from workshops and 
conferences and other meeting places where the impact of scientific 
computing on science is manifest and emphasis can be given easily to 
computations with the greatest impact. 

2) Theorists can saturate any computer, no matter how powerful with 
their simulation programs; once a computer is saturated, computing 
job queues lengthen and user productivity diminishes, which leads 
to pressure for more powerful computers. This fact makes it diffi¬ 
cult to determine the level of computing capacity to provide. For 
example, one may ask: why bother providing SI00 million dollars 
worth of capacity if, in the end, as many complaints will occur as 
if only S10 million dollars of capacity is provided? 

I propose the following strategy. The computing capacity made 
available to theory computing should be divided into two parts. A 
base level of computing support should be established, which every 
theorist is entitled to by virtue of being a qualified theorist. 

The base level would be supplied by computers which are cheap — so 
cheap that as many, of them would be supplied as necessary to prevent 
over-saturation of them. At the present time a national program 
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of reasonable cost could supply enough super minicomputers and 
attached processors to keep even with any level of demand that 
theorists could generate on them — if a particular theory group 
saturates one attached processor with legitimate research and 
teaching use they would be given one or two or four more, as needed. 
This procedure has the enormous benefit that computing time does 
not have to be allocated at the base level. As a result, students 
are free to try things out on base level systems without fear of 
exhausting an entire research allocation (this experimentation 
capability is exceedingly important for effective student training i n 
computational physics) and faculty and management time is not 
wasted on the allocation process. There should however be flexi¬ 
bility m the provision of base level computing - for example where 
existing research areas are already extensively tied to super¬ 
computers by existing software such as the Plasma Fusion energy 
system based at Livermore, there is no point in forcing changes. 

Many computing projects, when they are first begun, can be targeted 
equally effectively f or a supercomputer or for an attached processor 
despite the factor of ten or more advantage in speed of the super¬ 
computer. The reason is that when the factor of ten in computing power 
is converted into a grid spacing or the potential accuracy of a statisti¬ 
cal simulation, it is much less of a difference. Unfortunately, once the 
decision has been made to use the supercomputer level of power it is much 
more difficult for a user to retrospectively shrink a program from the 
supercomputer to a less powerful machine. It is important that theorists 
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hot vet using surercomouters be Dointed towards attached orocessors from 
the start — if this is done there will be very little loss in scientific 
productivity due to using the much cheaper attached processors instead of 
supercomputers for the base level computing. 

There is still an extremely important role for supercomputers. I 
would reserve them for a few key programs that require the entire capacity 
of a supercomputer. For example, a Cray X-MP could be 40 times faster 
than an Attached processor. This speed is most useful if there is just 
one program using the entire Cray, say for a week. These wholesale .lots 
of supercomputer time would have to be allocated; a possible mode of oper¬ 
ation would be as follows. Programs accepted for supercomputer runs would 
nave to meet three criteria: 

a) only programs ready to run on the supercomputer would be considered, 
and only if all necessary preparations that could be done at the 
based level are complete. 

b) any program accepted for runs on the supercomputer should have a 
reasonable chance of producing truly useful results in the 
allotted time 

c) any program accepted should have scientific priority; for example, 
a workshop might establish that a particular simulation is blocking 
progress and needs to be completed in preference to other projects 
in the same area. 

These criteria should be kept strict enough to avoid saturation of 
available supercomputers, so that once a program is accepted it is 
scheduled quickly, while the whole problem is still fresh in the user's 
mind. Tne advantage of this approach is that the scientifically most 
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important computations can be given the factor of 100 or more increase in 
computing power above base level that they often require for satisfactory 
completion. 

No method of supplying computing capacity will work unless computing 
capabilities at the base level increase rapidly. Simulation projects, 
once started require growing capabilities especially to allow determin¬ 
ation of errors; new levels of capability must be reached so that new levels 
of complexity can begin to be explored by simulation. The growth of 
theoretical physics will be stunted if computer capabilities do not grow. 

The workshop report proposes a major effort in computer development. 

I heartily endorse that effort; it is so important that I believe it 
should receive roughly half of all funds provided for the large scale 
scientific Simulation program. The computer development effort should 
have two parts. One part is to develop extremely powerful systems that 
would be vast upgrades for existing supercomputers. The other part should 
be to expand the capability of the base level of computing supplied for 
scientific use. The incredibly rapidly growing capabilities of VLSI 
technology should be used as far as possible for base level systems. The 
reason for this is that by heavy investment in state-of-the-art VLSI design 
and equally heavy investment in large scale manufacturing facilities, the 
unit costs of VLSI components can be reduced to almost 0, as the plunging 
prices of memory chips illustrate. For the base level of computing the 
most urgent need is to reduce the unit cost of single systems so that 
there is a maximum ability to keep on providing additional systems to the 
scientific community to prevent saturation and attendant loss of Droduc- 
tivity. 

The development of very powerful systems that are factors of 10C and 
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1000 and even 10° above the base level are. equally important; in the plan 
I outline these systems would be used to enable completion of only the 
most urgent scientific simulations, even when factors of 1000 or 10^ are 
needed to finish these simulations with adequate reliability. As 
scientific simulation grows generally in importance both for basic research 
and practical applications, the most urgent simulations will become far 
more important than the cost of the computers they require. 


III. Management and Manpower Issues 


The draft workshop report does not address two important practical 
issues. One is the problem of computer management, the other is the 
impact of a major computing support program on the national technical 
manpower pool in computing. ™ 

Good medium, and high level management for computing systems is 
extremely scarce, as most university people know from local experience. 

The most difficult role in a computing center is that of the middle 
level manager, who must cope with major problems coming from both above 
and below. In addition, whenever any large community of users and supoort 
personnel is organized, the community as a whole tends to become very 
resistant to change. Over the next ten years there are likely to be 
extraordiriSary changes taking place — from vector supercomputers to parallel 
systems to a great variety of special purpose systems, from FORTRAN to a great 
variety of higher level programming systems. Any program to deal with 
large scale scientific computing needs must continuously adapt and respond 
to these changes. 

In the plan I presented, the base level of computing would be provided 
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mostly by self-contained systems serving small, localized theory communit¬ 
ies, often within a single university department. Tne only support staff 
required is one system manager, perhaps aided by some undergraduate work- 
study students. This system manager's task is easily performed by eager 
young people: not much prior experience is required. These small units 
can adapt to change fairly easily when it is necessary to do so. 

The supercomputers, in the plan I described, would also have small 
user communities; these systems should also be easy and inexpensive to 
manage and to adapt to change. 

Strong management would be required, I think, for the following 
areas: 

a) bringing up initial production versions of major new computing 
systems - this involves innumerable software and consulting 
headaches which makes good management essential. 

b) a national high performance network 

c) major archives of a permanent or semi-permanent nature 

d) very major software packages receiving widespread use. 

I strongly urge that centralized management skills available to large 
scale scientific computing be targeted whenever possible to the areas of 
greatest need; in particular the base level of computing should be distri- 
buted and informally managed to the extent that local conditions permit. 

Any major program involving computing must be scrutinized for its 
impact on the nation's trained manpower pool, since any reduction in this 
pool can result in reduced economic activity, especially within the com¬ 
muting industry itself. The professional support personnel used as system 
managers or in central computing centers are drained from the overall 
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manpower pool. The impact will be less if these personnel move on to 
industry fairly quickly, but in the meantime train their successors. 

In contrast, undergraduate work-study programs, higher level software 
projects which help support computer science graduate students, and 
supported computer development projects in computer science departments 
help increase the manpower pool. In addition a basic function of the 
large scale scientific computing support effort is the training of graduat 
students in computational aspects of science. 
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Japan's Initiatives in Large Scale Computing 

Lawrence A. Lee 
National Science Foundation 

1. Japanese Supercomputer Project 

In July of 1980 Japan announced a supercomputer project aimed at creating a 
machine by 1989 capable of speeds at least three orders of magnitude higher than 
currently attainable, that is, a 10 gigaflop computer. Six major Japanese 
computer vendors (Fujitsu, Hitachi, NEC, Mitsubishi, Oki, and Toshiba), led by 
the government's Electro-Technical Laboratory (ETL), have joined in this project 
entitled National Super-Speed Computer Project (NSCP). The joint effort is 
organized as the Scientific Computer Research Association (SCRA). The project, 
started in January 1982, is scheduled for completion by 1989 and is funded by 
the Ministry of International Technology and Industry (MITI), with additional 
support from each of the six vendors. Total funding for this project is 
expected to be about S200M. 

The research is to be carried out in two subprograms simultaneously: 

(a) Development of new types of high-speed logic and memory elements. 
Researchers will consider a variety of possible alternatives to silicon 
semiconductor technology—including Josephson Junction elements. High 
Electron-mobility Transistors (HEMT) and Gallium Arsenide Field-effect 
Transistors—with the goal of producing processors with speeds in the 
40-100 mflops range. 

(b) Development of parallel processing systems incorporating arrays of as many 
as a thousand processor elements. 

Finally, the results of the two subprograms are to be combined, resulting in 
large-scale parallel systems which use 40-mflops to function at speeds of 10 
gflops or even faster. 

For the sake of comparison, the following are operating speeds and in-service 
dates of several existing and planned U. S. supercomputers: 

The CRAY-1: 88 mflops (1976) 

The CDC Cyber 205: 400 Mflops (early 1981) 

The CRAY-2: about 800 mflops (late 1983 or early 1984) 


2. Japanese 5th Generation Computers 

MITI is funding another project called the National Fifth-Generation Computer 
Project; this project will start in April 1982 and run for at least ten years. 
MITI and vendors will provide some S500M for this project. Whereas the NSCP is 
based on extensions of current technology, the fifth-Generation Project is to be 
a revolutionary approach to computer design based on artificial intelligence. 

The overall project goal is to design information processing systems to deal 
with the basic social problems Japan foresees for itself in the 1990's (such as 
low productivity in primary and tertiary industries; international competition; 
energy and resource shortages; and a rapidly aging population). 
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Among the specific applications set as eventual goals for the project are: 

(1) A machine translation system automating 90% of natural-language 
translation (e.g., between Japanese and English). 

(2) A consultation system capable of functioning as a small interactive 
reference library for workers in various specialized fields. 

(3) A system capable of automatic or nearly-automatic creation of software from 
specified requirements, with little or no need for computer sophistication 
on the part of the user. 

This is a highly ambitious attempt to change the whole domain of operation 
of computers in society from "Data"—abstract bits of information stored in the 
form of electrical impulses and manipulated by machines without regard for what 
they might signify to Knowledge"—something much more like people's everyday 
use of the word information,' and much more accessible to people who do not 
have special training in computer/science. 

The project is described in terms of certain "key words," including: artificial 
intelligence, parallel processing, inference mechanization, knowledge-base, 
relational data-base, relational algebra, software-development systems, 

VLSI-CAD, machine translation, and Prolog-like languages. 

One approach to building such new machines is to carry on the work of the past 
three decades in increasing storage capacity and operating speed. The fifth 
generation project in its final approved form contains little if any work of 
this kind; rather, administrative mechanisms will be set up to allow the fifth 
generation researchers to take advantage of exploitable new hardware 

t * e ^ e l£* :)mentS as are ac * neve( l i°> for example, the supercomputer project, 
and the next generation basic industries semiconductor project. The hardware 
approach that will be taken in the fifth generation project will focus on 
exploring new computer architectures designed specifically for efficient 
operation of the knowledge base" fifth generation software. 


3. Japan's Vector Processors 


In addition to the supercomputer and 5th generation projects, each of the 


affect future supercomputers 

The 


vendors has one or more internal projects that will ___ 

in the areas of architecture, software, algorithms, and device^technology" 
company project that appears closest to completion is the Fujitsu Vector 
Processor (VP); this computer is based largely on the same technology as the 
Fujitsu M380/382 (the M380/382 is to be delivered this year and will exceed the 
performance of the announced IBM 3081 products by about a factor of 1.5). The 
Fujitsu Vector Processor is intended to compete with the Cray-1 and will (if all 

goes well for Fujitsu) exceed the performance of the Cray-1 by up to a factor 
o f 5. 
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WORKSHOP ON LARGE SCALE COMPUTING FOR SCIENCE AND ENGINEERING 
POSITION PAPER 
PREPARED BY 


Robert G. ‘Gillespie 
University of Washington 


June 18, 1982 
Introduction 


The following issues dealing with the Workshop On Large Scale 
Computing for Science and Engineering were developed during 
discussion with a group of faculty members from the University of 
Washington. The disciplines involved included Chemistry, Physics, 
Atmospheric Sciences, Oceanography and Computer Science. 

Issues 

1. ACCESS 

A strategic option to address the fundamental question of 
providing access to large scale computational facilities would be 
linkage of the existing facilities with major research universities^ 
It was assumed that high band width communication facilities, such as 
can be provided from satellite stations, would be provided. The 
establishment of these nodes would significantly aid the resource 
sharing of the class VI facilities and also would have other 
secondary, effects by making the communication effort transparent. 
Essentially, the nodes could reduce the friction that limits use of 
existing information resources. This option should be viewed as a 
valuable alternative when compared to adding one more CLass VI 
computer. 

2. COMMUNITIES OF INTEREST 

Scientific communities tend to be organized around discipline 
areas. The opportunities for resource sharing—-ideas, software, 

databases-along with improved effectiveness associated with 

computer support for reasonably homogeneous communities, should 
command attention when options for approach are considered. The most 
successful of the shared scientific large scale systems have been 
organized along discipline lines. This approach simplifies the 
allocation because, with a homogeneous community, allocation 
decisions can be made by members of the scientific community. 

A clearly perceived need should exist before a new facility if 
funded and built. This is another reason that discipline oriented 
centers (e.g. NCAR) have been successful. While other arrangement AP 
such as regional centers might work effectively, the need should be 
clearly seen. 
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3. FUNDING 

choo S fng d i£!v f ?o p^dfaddiSf? 1 ? 9 f °f ld be approached by 

riska £ra5^ti„g P t?e rlslu?=ei 2Jai^e ™ S Choi « 

converting the problem of nrovidinn I? le nationally as well as 
strictly entrepreneurial model^ dln9 ““Puting facilities to a 

4. PILOT PROJECT RISKS 

can be 1 a° t effJctive a Say°?o h deiel^ support " an i' Projects, and 

computing systems involves more resour?esthanth»f °f- lar?e scale 
to successfully achieve effective . an th ? fun ding or access 

undertake personal investmentinief^, investigator must 
existing codes which took in . ea tning new systems, redeveloping 

Pilot (sometimes ^synonym for underfund^r” 310 stablU «- a 

the success, some of the elemenflof tuff pro;|ect “y never achieve 
observations of the experience with th^n t 90nC f rn were generated by 
Computational Chemist™ T^f f be National Resource for 

pilot phase should be carefully eJaluftfd‘ n fo 3 “" eSted t that hava a 
not be self fulfilling prophesied Xu u ^. lnS “ re that - they “ iU 

5. FEDERAL DISINCENTIVE PROGRAMS 

strategy and*approaches to pol ! ci ® s aff ects the 

investigators. The lack of a funding universities by individual 
ongoing computer use and operaLnc ™* cban ^ f ° r long term ' 

small and intermediate computer svstemJ fc ® nds P ush acquisition of 
funds in a grant or conS a e In’ f tlally the ca P ita l 

computational resources. UnfortunatelJ thi«, for 

not necessarily either meet \ th suboptimization does 

Federal resources whtre rtlourctshltinZ 9 ? a1 ?,° r • draw to 5ether 
appropriate. sharing of facilities would be most 

6.PROCUREMENT CYCLES AND TECHNOLOGICAL CHANGE 

when alternatives for^cquisition of e v i° n9 i Federal Procurement cycles 
considered. Cycles of over fiie vefrs ^ ^ SCale s * stems a ^ 
present state of obsolescence at ESTV ^ ? een observed and the 
the acquisition of out of date equipment^ laboratories could mean 
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I. Supercomputers 

By “supercomputer" I mean the computer with the largest memory and the 
fastest CPU currently available commercially. At the moment (September 1982), 
the two major American supercomputers are the CRAY-1 and the CYBER 205. Such 
machines possess 1—4 million 64—bit words in the central memory with a rela¬ 
tive capacity of more than 4 CDC 7600*8 (the equivalent of 6 CYBER 175's). 

[See Appendix B for rough conversion figures.] 

The CRAY X-MP machine, which becomes available in one year, will yield a 
throughput increase of - 3-5 over the CRAY-1. It also offers a 10 gigabits/sec 
transfer rate to 32 million (64-bit) words of external solid state memory, in 
addition to the 4 million words in central memory. This would make it equiva¬ 
lent to 30-50 CYBER 175s. The above figures are used to "set the scale" for 
what universities should be thinking about in the next few years, and do not 
constitute an endorsement of any manufacturer. 

The current distribution of supercomputers (Appendix A) virtually ex¬ 
cludes American universities. As of mid-1982 there are ~ 50 supercomputers in 
the world. In this country the national DOE weapons labs have the largest 
concentrations of them. Lawrence Livermore National Laboratory (LLNL) and Los 
Alamos Scientific Laboratory (LASL) each have four CRAY-1*s, or 1/5 of the 
world supply. Specialized scientific groups have access to CRAY'S at National 
Centers, e.g. the National Center for Atmospheric Research (NCAR) and the Mag¬ 
netic Fusion Energy Center (MFECC). Finally, a number of major corporations 
have acquired CRAY'S, e.g. Bell Labs and Exxon. European scientific centers * 
have also begun using supercomputers, e.g. the Max-Planck Insltute (MPI) for - 
Physics and Astrophysics in Garching-bei-Munchen, West Germany. 

The first arrival of supercomputers at American universities has occurred 

in the last year. Colorado State University and Purdue University have each 

bought a CYBER-205, while the University of Minnesota has obtained a CRAY-1. 

At least a dozen universities are presently considering whether to purchase 

supercomputers either individually or in regional groups. 
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II. Flight From the Universities 

The drought of computing power in the universities over the last five 
years has resulted in a a major drain of highly talented researchers from the 
universities to the sites with supercomputers. I perceive two stages in this 
process: first, a flight to the weapons labs (LLNL and LASL), and second, a 
flight to foreign computing centers. Host of this section is anecdotal know¬ 
ledge and represents my narrow sphere of expertise. I imagine similar stories 
can be told by others in different areas of research. 

Much of the work in computational astrophysics has been done at LLNL and 
LASL during the last decade. Areas of numerical research include: super¬ 
novae, general relativity (black holes, neutron stars, gravitational waves, 
cosmology), nonspherlcal accretion onto compact objects, jet formation, star 
formation, and radiation astrophysics. Many scientists from universities who 
wanted to work on these frontier fields were forced to go to the national labs 
to gain access to the largest computers. As a result of this limited access, 
as well as the standard shortcomings of American universities (i.e. low sala¬ 
ries, lack of job openings, etc.), many of these people became lab employees. 
Others remained consultants to the labs so they could do their computing while 
remaining in the universities. 

More recently, computational astrophysicists have been going to MPI in 
Munich, West Germany to do their computing. This is because the MPI CRAY-1 is 
devoted to physics and astrophysics, not to weapons design. Even scientists 
employed at the weapons labs can often get more CRAY time for their astrophys¬ 
ics projects at MPI than they can at LLNL or LASL. In the last two years the 
following astrophysicists have spent extended visits at MPI: David Arnett (U. 
of Chicago), Peter Bodenheimer (U. of CA.- Santa Cruz), Dick Miller (U. of 
Chicago), Michael Newman (LASL), Michael Norman (Ph.D. from U. of CA. - 
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Davis), Larry Smarr (0. of IL.), and Paul Woodward (LLNL). A similar list ex¬ 
ists for theoretical chemistry. There are about 100 visitors to MPI per year, 
of which 20 are heavy CRAY users. 

Currently, the Japanese are furiously publishing work in the area of 
numerical hydrodynamics. Their major fields of attack so far are protostars, 
stellar evolution, accretion onto compact objects, radio jets, and general re¬ 
lativity. They are limited by their currently available computers, but within 
one to two years supercomputers will come on-line. At that point it is highly 
likely that Japan and Germany (already on-line) will take over from the U.S. 
the scientific leadership in numerical astrophysics. 

Besides the obvious loss of talent from the universities to the weapons 
labs and foreign institutions, what are the other costs American universities 
must pay for lacking supercomputers? The primary problem is that numerical 
science must be done in the "crash program" mode. A visit to a supercomputer 
site is limited in duration: typically a few days to a few months. Thus one 
is under great pressure to complete a project in a short period of time. This 
means long hours (100 hours per week is not uncommon), great fatigue, little 
time for thinking or literature searching, and no time to discuss the project 
with colleagues. Furthermore, there Is not the diversity of colleagues to 
confer with that one has nurtured in one's university. In many cases, one 
can t even look at the output of the calculation until one has left the site. 
This is not good scientific methodology. 

Clearly, it would be much better to have the supercomputer (or access to 
one) at the home institution of the scientist. Then he could spend all year 
in compute, think, read, talk, recompute" mode. That is, he could do a cal¬ 
culation, produce output graphically, thoughtfully look through it, realize he 
needed different functions plotted, compare his work with previous efforts, 
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discuss his ideas with other American-based collegues, do an analytic side- 
calculation, understand more deeply what the physics is, and then decide what 
to compute next. He has access to a friendly and comfortable local computing 
system instead of having to learn an entirely new, foreign system in a few 
days' time. He has graduate students, secretaries, his own office and li¬ 
brary, and a familiar circle of scientific colleagues to talk with. All of 
these factors are important for good scientific research. Finally, only the 
most hardy souls with the most forgiving families can socially, psychological¬ 
ly, and physiologically withstand the constraints Imposed by computing ses¬ 
sions at distant supercomputer sites. Having supercomputers available to uni¬ 
versities would bring many more top scientists into the numerical science 
revolution. 

III. Frontier Science and Supercomputers 

The above arguments do not pinpoint why supercomputers, rather than say 
many small computers, are essential in areas of frontier science. Perhaps the 
most succinct statement is that attributed to Edward Teller. "Teller's Rule" 
is that the most challenging frontier problems in science require at least 100 
hours on the fastest computer available. As computers become faster, the 
knowledge gained on previous machines enables one to fruitfully attack the 
next level of Complexity. Currently, there is virtually nowhere in the world 
where one could get this much CRAY time in a year as on off-site visitor. 

One could make the argument that such problems could be attacked on small 
ho6t computers attached to dedicated array processors. Indeed, fundamental 
work in quantum field theory and astrophysics has been done in this mode by 
some groups (e.g. the Cornell array processor users group). However, to date 
most computational astrophysicists prefer to have the large memory, high 
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speed, fast turnaround of a supercomputer. The point Is that the scientist, 
should be able to choose the tool which best enables him to do his science on 
his. timescale. Currently the answer to this choice Is being masked by the 
physical difficulty of gaining access to supercomputers. Let me turn to a few 
examples of the type of fundamental science that Is currently being done on 
supercomputers, again let me stress that I can only relate areas of research 
I have personal familiarity with. The Press »SF Report "Prospectus for compu¬ 
tational Physics," 1981, details many more areas. Also, some of this work la 
work that wi ll have to be done on supercomputers to get a reliable answer, but 
which is currently forced to be done on lesser computers (e.g. CDC 7600 or 
CYBER 175) becauae of lack of access to supercomputers. 

My set of examples are a list of - 60 scientists at the University of 
Illinois who. I have talked to about the desirability of obtaining a supercom¬ 
puter at the University of Illinois. These are researchers who would either 
actively use the supercomputer or bring In associates who would. The on-line 


description of areas of potential supercomputer research la augmented by more 

detailed statements of research.intent In Appendix C. The clear conclusion 1. 

that there la a tremendous breadth of areas represented st only one major uni- 
versity. 
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POTENTIAL SCIENTIFIC PROJECTS 
University of Illinois 


CRAY X-MP Facility 
(as of 11/1/82) 


- Experimental Particle Physics 


Physics Department 

* Richard Brown (Prof.) + 9 other fac¬ 

ility 

* Haldan Cohn (Res. Asst. Prof.) 

* John Dow (Prof.) 

* Hans Frauenfelder (Prof.) 

* John Kogut (Prof.) 

* Barry Kunz (Prof). 

* Fred Lamb (Prof.) 

* V.R. Pandharipande (Prof.) 

* David Ravenhall (Prof.) 

* John Stack (Assoc. Prof.) 

* Michael Stone (Assist. Prof.) 

* H. W. Wyld (Prof.) 


- Evolution of Stellar Systems 

- Magnetohydrodynamical Accretion 

- Electronic Materials 

- Dynamics of Biological Molecules 

- Solving Quantum Chromodynamics 

- Quantum Chemistry of Solids 

- Radiation Hydrodynamics of Neutron Stars 

- Quantum Liquids and Solids 

- Nuclear Structure and Scattering 

- Quantum Field Theory 

- Simulating Quantum Field Theory 

- Lattice Quantum Chromodynamics and Turbuler 

Simulation 


Astronomy Department 

* Joan Centrella (Post. Doc.) 

* Helene R. Dickel (Res. Assoc. Prof.) 

* John Gallagher (Prof.) 

* Icko Iben (Prof.) - 

* Steven Langer (Res. Assoc.) - 

* Telemachos Mouschovias (Assoc. Prof.) - 

* Larry Smarr (Assoc. Prof.) 


* Lewis Snyder (Prof.) 

* James Truran (Prof.) 


Large Scale Structure of the Universe 
Radio Interferometric Spectral Line Ana^M 
Radiation Hydrodynamics of Molecular Cocoon 
Optical Astronomy Instrumentation 
Stellar Evolution 
Accretion Onto Neutron Stars 
Star Formation 

Numerical General Relativity 
High Resolution Finite Differencing of Hydr 
dynamics 

Chemical Astrophysics 
Hydrodynamics of Novae 
Nucleosynthesis Networks 


Chemistry Department 

* David Chandler (Prof.) 

* Clifford Dykstra (Assist. Prof.) 

* Douglas McDonald (Prof.) 

* Donald Secrest (Prof.) 

+ Peter Wolynes (Assoc. Prof.) 


Quantum Dynamics of Molecules 
Electronic Structure Theory 
Intramolecular Dynamics of Molecules 
Molecular Scattering Theory 
Monte Carlo and Renormalization Group in 
Quantum Chemistry 

Simulation of Solltons in Polymers 


All 1-12 
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Computer Sciences 

+ Roy Campbell (Assoc. Prof.) 

* Charles W. Gear (Prof.) 

* William Kubitz (Assoc. Prof.) 

* Daniel Gajski (Assoc. Prof.) 

* David Kuck (Prof.) 

* Ahmed Sameh (Prof.) 

* Paul Saylor (Assoc. Prof.) 

* Dan Slotnick (Prof.) 

* Daniel Watanabe (Assoc. Prof.) 


- Software Engineering 

- Numerical Software 

“ VLSI Design Automation 

- Supercomputer Software & Algorithm Development 

m H 

- Iterative Solution of Large Systems of Linear 

Algebraic Equations 

- Supercomputer Design & Application 
“ Simulation of Semiconductor Devices 


Atmospheric Sciences 

* Stan Kidder (Assist. Prof.) 

+ Yoshimitsu Ogura (Prof.) 

+ Su-Tzai Soong (Assoc. Prof.) 

+ Kevin Trenberth (Prof.) 

+ John Walsh (Assoc. Prof.) 

* Robert Wilhelmson (Assoc. Prof.) 


- Forecasting with Satellite Data 

” Simulation of Convection & Turbulence 

- Simulation of Mesoscale Convection Systems 

- Climate Modeling 

- Sea Ice Modeling 

- Dynamics of Severe Storms 


Chemical Engineering 

* Thomas Hanratty (Prof.) - Turbulence 

Mark Stadtherr (Assoc. Prof.) - Chemical Process Design 


Electrical Engineering 

* Jacob Abraham (Assoc. Prof.) 

* Sid Bowhill (Prof.) 

* Edward Davidson (Prof.) 

* Karl Hess (Prof.) 

* Nick Holonyak (Prof.) 

* W. Kenneth Jenkins (Assoc. Prof.) 

* Michael Pursley (Prof.) 

* Timothy Trick (Prof.) 


- VLSI System Design 

- Three-Dimensional Atmospheric Modeling 

- Computer Architecture & Reliability 

- Simulation of 3—D Submicron Semiconductor Devic< 

- Quantum-well Heterostructures 

- Inverse Problems in Tomography, Synthetic Aper¬ 

ture Radar Images, and Sonar Beam Forming 
Performance Evaluation of Communication Systems 

- Two-Dimensional Simulation of Electronic Device/ 


Mechanical and Industrial Engineering 
+ A. L. Addy (Prof.) 

+ Richard Buckius (Assoc. Prof.) 

* Herman Krier (Prof.) 

* Frederick Leckie (Prof.) 


- Gas Dynamics 

- Radiation Heat Transfer 

- Hydrodynamic Shock Physics 

- Damage Mechanics 


Theoretical & Applied Mechanics 
+ Ronald Adrian (Prof.) 

* Lawrence Bergman (Assist. Prof.) 

* Robert McMeeking (Assoc. Prof.) 


Civil Engineering 

* Robert Haber (Assist. Prof.) 

* Leonard Lopez (Prof.) 


- Comparison of Simulated Turbulence with 

Experiment 

- System Dynamics 

- Fracture Mechanics 


- Structural Optimization 

- Structural Mechanics and CAD/CAM 
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Aeronautical & Astronautleal Engineering 
* S. M. Yen (Prof.) 


-* Rarefied Gas Dynamics 
- Computational Fluid Dynamics 


Anatomical Sciences 

* Jay Mittenthal (Assist. Prof.) - Biomechanics of Cell Sheets 


Genetics and Development 

* Robert Futrelle (Research Scientist) - Simulation of Living Cells 


Physiology and Biophysics 

* Eric Jakobsson (Assoc. Prof.) - Emergent Oscillatory Properties of 

Neural Networks 


Accountancy 

* James McKeown (Prof.) 


- Multiple Time Series Analysis 


* - spoken to by L. Smarr 
+ ■ spoken for by * people 




Wilson 



Floyd R. Newman 
Laboratory of Nuclear Studies 

Cornell University 

Newman Lab. Cornell Univ. 

Ithaca. New York 14853 
607-256- 5169 


July 12, 1982 


To: The Lax Panel 

From: K. Wilsony v G,v*U~ 

Re: Priorities for Funding Current Needs 

I suggest that the priorities for funding current needs, the subject 
of panel 2's discussion, be stated as follows. The basis for this plan 
should be the "Prospectus for Computational Physics" report (generalized 
to all areas of scientific simulation). The priorities proposed below are 
intended to implement the plan of that report for an effective national 
network of national, regional, and local computing systems. There should 
be heavy emphasis on the importance of the network as opposed to any 
particular site on the network. There should be minimum standards set 
for access and ability to use remote capabilities on the network, which 
most governmentally-funded researchers should be entitled to. The first 
priority should be to achieve this minimum standard for any researcher 
seriously involved in large scale scientific computation. 

Secondly, it should be stated that none of the proposed budgets can 
_ come close to providing adequate cycles for scientific computing in uni¬ 
versities, if only present-day or announced future large scale systems are 
used. It is not possible within this funding to provide all university 
researchers with the computing capabilities presently available (per 
researcher) at Los Alamos and Livermore. Besides which the capabilities 
even at Los Alamos and Livermore are grossly inadequate for current needs 
in basic research. They are even more inadequate to train students for 
the systems they will use professionally when they graduate (because of 
rapid technology development, the systems our present students will use 
after graduation will make the Cray-1 look like a desk calculator). Hence 
it is essential that rapid technology development be given very high 
priority; the proposals for providing cycles on current systems are only 
a very inadequate stopgap measure. 

Therefore, 1 suggest the funding priorities stated below. This 
proposal is in addition to the proposals to supporting technology develop¬ 
ment, which I do not address here because panel 3 already has a good 
proposal worked out. The funding for technology development should have 
equal top priority with minimum network access. 

1) Minimum standard network access from universities should include 

a) hardware and software to attach to network (to the extent 
not provided by DOE or other sources; also access must not 
be restricted to specific areas of basic research) 

b) Terminals in adequate number 
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c) Local graphics capabilities — there is no centralized 
substitute for local graphics 

d) modest dollar grants for local operating expenses and remote 
access computing charges 

For this purpose, grants should be available for individual research 
groups or consortia, in the range of $10 to $50 K per principal 
investigator. Unless this need for access is met, the concept of 
the network is a joke, 

2) A class VII ( not class VI ) national facility, with heavy emphasis 
on selecting a site that can achieve full production operation of 
the system rapidly . This is essential because of the rapid obsol¬ 
escence of supercomputers that we are likely to face for the 
forseeable future. 

3) Support for specialized capabilities at specific network modes 
(for example — special hardware and software offerings, or centers 
for specific disciplines). 

With a full budget I would give equal weight to the three types of support, 
but with less than a full budget I would strictly adhere to the priorities 
even if that meant no funding at all for low priority items. Recall also 
that technology development has (in my view) equal priority with minimum 
network access. 


KW/jmm 
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"Yale University 

Chtmury • I'.O H„ x . ,V„, • « ."Wi.nuit • M,)H 

Kenneth li. Wiberg 
203-436-2443 


May 26, 1982 


Dr. Richard Nicholson 
Division of Chemistry 
National Science Foundation 
Washington, D. C. 20550 

Dear Rich: 

I recently returned from a JSPS sponsored 4 week trip through Japan. As 
you well know, their research work in chemistry has gotten to be quite impress¬ 
ive. I was surprised by the quality of their instrumental facilities. Every place 
I visited had a high field nmr spectrometer in the order of 300-500 MH Zi and ex¬ 
cept for FT-1R, their oilier instruments were comparably good. At the present 
time, I think Japan and the U.S. are at about the same level with regard to gen¬ 
erally used instruments. 

There is one aspect of instrumentation in which they appear to be well ahead 
of us.. At the Institute of Molecular Science, there are two CDC-7600 computer 
and Morokuma is planning for a new computer on the order of a Cray 1 in 1-2 
years from now. At the National Chemistry .Laboratory for Industry, there is 
a large Japanese computer with fiber optics links to the laboratories for high 
speed data transfer. The chemists at Tsukuba University (located a short dis¬ 
tance away) have access to this computer. The University of Tokyo is planning 
to get a computer with the capabilities of a Cray-1 or Cyber-205. Tohuki Univ¬ 
ersity and Hokkardo University (on different islands) have their computers con¬ 
nected via a fibe^r optics link. 

Every university I visited lias a fairly large computer operated witii a rather 
low user charge (“ Jf/s.or 5100/ hr. with discounts for large users - overnight 
runs, etc.). In the U.S. , no university can match the com puting power avail¬ 
able to Japanese chemists. Lvcn LBL and 13 N L have only one 7 600 each, whe re 
as The Institute of Molecular Science has two - for a smaller group of in-house 
use rs. 

Japanese chemists have not, for the most part, come to reali/.e the potential 
impact of computational methods on chemistry. Experience so Jar indicates that 
they can learn quickly - and they could essentially take over computational chcm- 
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istry. NRCC was a feeble effort in comparison to what Japanese chemists 
already have - and even that was abolished. I am concerned about the futu* 
ofU.S. chemistry. 


Sincerely, 



KBW :jl 



Brenner 




Fermilab 


Fermi National Accelerator Laboratory 
P.O. Box 500 • Batavia, Hlinois • 60510 


-June 23, 1982 


Professor Peter P. Lax 

Courant Institute of Mathematical Sciences 
New York University 
251 Mercer Street 
New York, NY 10012 

Dear Peter: 

in tL W Mcc Ple 5 S nnn t0 be ab1e to P art icipate, at least in part, 

for t SriPnrp 3 ^ d H D p D sponsored Workshop on Large Scale Computing 
for Science and Engineering. Unfortunately, I had to leave for 

another meeting so that I was not able to attend Tuesday's 
Plenary Session. I did, however, have an opportunity in the 

Schwartz t0 COntribute a bit to the working group chaired by Jack 

From my observations and efforts Tuesday morning, I'm 
convinced it s not going to be easy to put together a report 
thPcp P r!Sr r y projects all of the concerns and the urgency of 

C u nC f r u S that . were expressed by many people participating 
n the Workshop. Without the benefit of seeing the body of the 

attempt l\ Vfl ^quefted I'Me Tn 

JnSIJ- at a f ew sentences which might become part of a preamble 

inrinHpH e - S0 Th ° f the a * eas ° f com P utin 9 whose needs arenot 
included in the scope of the Workshop's attention. 

Since the advent of the modern computer, many science and 
engineering disciplines, taking advantage of the new 

Stri5« 1 ' ieS / hat ' these tools make possible, have made enormous 
strides in advancing their fields. The breadth and depth of 
functions to which these computing tools, in their various 
embodiments, have been p.ut to use are enormously varied. 
Furthermore, even with the rapid improvement in performance and 

need^ofHfh^sciencp °Y heS ? tool s , "thTcolpuSati o 1 

fastjYthar^indust ry i h#Ve 

"One of the most important areas where the needs have far 
outstripped the capability of commercially available tools is in 
the large scale mathematical computing area, typically 
simulation of phenomena in science and engineering. Here there 
re r.!?,! 0 ?«•«•':* of ■•S.lt.d. improvement In 

addressed by ;hn y Worksnop." r,ma '' ,,y U ’ S " eed Kh1ch has been 
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"It should be noted, however, that there is yet another 
class of problems, with needs for large scale computers, likely 
to require different solutions. These needs also are not met by 
commercially available “systems by two and three orders of 
magnitude in computing power. These problems, associated with a 
number of areas of experimental physics, typically have 
relatively large amounts of input and output in association with 
a major amount of computational requirements, characterized by a 
large number of branching operations. Examples of these 
problems are found in the high energy physics and the magnetic 
energy fusion communities when they analyze their data. 

Although some of these problems are suited to special 
purpose-type processing machines, in most cases large scale 
general-purpose computing engines of appropriate architecture 
(very likely different from that which will serve the simulation 
class of problems) are more practical. In any case, these 
problems which may require different solutions than those under 
consideration here are not further discussed in this report." 

Although I am sure that these three paragraphs will require 
much editing, I do hope they give you a framework from which may 
evolve an appropriate preamble. 

If there is anything else that I can do to help in the 
report of the Workshop, please feel free to call upon me. I do 
hope that I will see a copy of the end product. 


Sincerely yours, 



ft. E. Brenner, Head 
Computing Department 


AEB: 1 p 
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WHY THE U.S. GOVERNMENT SHOULD SUPPORT RESEARCH 
ON AND DEVELOPMENT OF SUPERCOMPUTERS 

Bill Buzbee 
Computing Division 
Los Alamos National Laboratory 


OVERVIEW 

Supercomputers are the largest and fastest computing systems available at any 
point in time; thus, the term supercomputer encompasses hardware, software, 
and supporting equipment. This document asserts that supercomputing is 
important to U.S. Defense and advocates government action to maintain U.S. 
self-sufficiency and leadership. Consequently, we will 

o review the role of supercomputers in science and engineering; 

o assess their importance to the U.S. and, thus, the risk incurred if 
another country assumes leadership; and 

o discuss why the private sector is unlikely to commit the investment 
necessary to maintain U.S. leadership. 

The proposition is that government-supported research and development is 
required to lower technical/market risks to a point where private industry 
will commit to product development. 


ROLE OF SUPERCOMPUTERS IN SCIENCE AND ENGINEERING 

Basically, science and engineering consists of three activities: observation, 
hypothesis, and prediction. Observation involves the collection of data about 
physical phenomena, often under controlled conditions. Hypothesis is the 
formulation of theory to explain observation and typically yields complex 
mathematical models. Prediction is achieved by applying these models to real 
world problems. In all three cases,•computers enable the scientist engineer 
to treat complexity that is not otherwise tractable, and, in so doing, the 
scientist engineer gains insight and understanding. Appendix I illustrates 
this point. Thus, acquisition of knowledge is the single most important 
benefit obtained from computers. 

The amount of complexity that a computer can treat is limited by its speed and 
storage. If the computer cannot handle the desired complexity, the model is 
simplified until it is tractable (Appendix II contains an example of 
simplification). This is why scientists constantly press for bigger and 
faster computers--so that they can add complexity (realism) to their 
calculations. When a new supercomputer becomes available, it is analogous to 
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having a new telescope, because with it the scientist engineer will see things 
hot seen before and understand things not understood before. 

In general, many important problems cannot be treated analytically. Computer 
modeling is the only way to cope with them. 


IMPORTANCE OF SUPERCOMPUTERS TO THE U.S. 

State-of-the-art supercomputers are manufactured by two companies, both U.S. 
Fifty of their supercomputers are in operation worldwide—38 in the U.S., 10 
in Europe, and 2 in Japan. This is a small market, even worldwide. Of the 38 
in the U.S., 25 are in government laboratories. 

Current applications of supercomputers include 

design and simulation of very large-scale integrated (VLSI) circuits, 

design of nuclear weapons, 

nuclear reactor research, 

fusion energy research, 

directed energy weapons, 

intelligence, 

aerodynamics, 

atmospheric science, 

geophysics, 

petroleum engineering, 
electric power distribution, 
oceanography, and 
fluid dynamics. 

Potential applications include automotive design and manufacturing. 

Electronics, nuclear weapons, and energy production are major components of 
U.S. defense. Design and simulation of state-of-the-art VLSI devices, nuclear 
weapons, fusion reactors, and nuclear (fission) reactors require state-of- 
the-art supercomputers. As discussed in Appendix III, progress in each of 
these areas may be paced, even bounded, by progress in supercomputing. The 
same is true for many of the above applications. Further, other important 
applications, such as information analysis and software engineering, may be 
paced by progress in fifth generation technology (see Appendix IV). Thus, 
leadership in supercomputing is fundamental to U.S. defense and to U. S. 
technology in general. 

RISK IF ANOTHER COUNTRY ASSUMES LEADERSHIP 

Should another country assume leadership in supercomputing, U.S. defense and 
technology will depend on access to computers of foreign manufacture. This 
presents three risks: 
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1. Currency of Access. If other countries consume the first two years of 
production then U.S. scientists will be denied use of these machines and 
U.5. technology could eventually lag theirs by that amount. 

2 ‘ u Cn i al ° f Access ‘ In this worst case, U.S. technology could be 
handicapped until domestic sources are developed. 


3. 


Computing Technology. Development of supercomputers has always 
contributed to development of other computer systems. If another 
assumes leadership, we may lose these benefits. 


country 


These risks are unacceptable. 


WILL THE PRIVATE SECTOR MAINTAIN LEADERSHIP? 

their^n^/^* ^ t i C K SPeed ° f su P ercom P ute « ^ diminishing. In the ’60s 
their speed increased by about 100, in the '70s by about 10; in the ’80s it 

by 1CSS thaD 10 ‘ The P roblem is tha * conventional computer 
Drona ** approachin 8 fundamental limits in speed imposed by signal 

developments dissipation. Continued progress requires revolutionary 

t«hiol«v Ml TP"" archltecture that will affect all aspects of computer 
technoiogy [1] Basic research is required before full commercial development 

exploit ^SI ^ mUSt diSC ° Ver the ri 8 ht kind of architectures, ways to 
research^!™ instruction, and suitable software for them. This 

, . * h 11 necessitate collaboration between industry, government 
laborT rieS ’ acad emi a--industry knows how to build computers, government 
laboratories understand the computational requirements of large SCa le loTels 
and academia has innovative people. moaeis, 

HistoricaHy, the cost of developing a supercomputer in conventional 

comb H beCn $10 ° million (Redware and software). This cost has 

combined with the relatively small market to keep capital and resource-^ 

U.S. companies out of supercomputer development. It must be recognized that 

c n, r a " Ce n n fflarket h3S bCen P^^lly due to a lack of foreign 

will be s°h't ^ i C f St ° f devel °P in 8 a new supercomputer in a new architecture 
will be substantially greater; also the risk of market failure will be 

the private sector is unlikeirtrcarry^ut^he research^rdeveir 1 ' ’ 

government t0 U ' S - lea dership. The proposition advocated here\s that 

g ernment-sponsored research and development is required to identify and 

”::: av s:r ° , 1 d r lop :r tthat are ukeiy « «c tal c" 

" th ' riSk “ * •>» i “ —« the private 


CONCLUSION 


Computers are fundamental to the advancement of technology. This is 
particularly true of supercomputers because we use them to do investigations 
not previously doable. As we enter en ere where other countries recognize 
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their value and undertake their development and manufacture, the U.S. will be 
wise to maintain self-sufficiency and leadership in supercomputing technology. 


SUMMARY 

1. Supercomputers aid in the acquisition of knowledge. 

2. Progress in U.S. defense technology is often paced, even bounded, by 
progress in supercomputing technology. 

3. Dependence upon foreign sources for supercomputer technology poses 
unacceptable risk. 

4. Conventional supercomputers are approaching their maximal performance 
level. Continued progress depends on development of new architectures. 

5. The private sector may not assume the cost and risk of developing new 
architectures. 

6. Because self-sufficiency and leadership in supercomputing are imp.ortant to 
U.S. defense, the U.S. Government should fund research apd development 
that will lower the technical/market risk to a point where private product 
development becomes feasible 


APPENDIX I 
COMPUTER SIMULATION 

After the Three Mile Island accident, the Presidential (Kemeny) Commission and 
the Nuclear Regulatory Commission asked Los Alamos to analyze, the early stages 
of the accident using computer simulation. Excellent agreement between 
computed and measured temperatures and pressures provided confidence in the 
computed results. But, of particular significance to this report, the 
computations provided information not otherwise available 12].; namely, they 
revealed the source of hydrogen that eventually caused so much concern. Thus, 
the use of computers provided new understanding about the accident. 

Because of this demonstrated ability to simulate reactor behavior, people have 
suggested that this computational tool be made available to reactor operators, 
so that in future accidents it can be used to predict the merits of remedial 
alternatives. Unfortunately, these calculations require hours to perform on 
state-of-the-art supercomputers. For practical use by a reactor operator, the 
computation would have to be done in at most a few minutes and that 
necessitates a computer at least 100 times faster than any available today. 
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APPENDIX II 

SIMPLIFICATION OF COMPUTER MODELS 

One approach to producing energy from fusion uses magnetic confinement. Many 
of the associated devices contain a torus (a "doughnut" shaped container). 
Plasma is confined in it and then magnetically compressed and heated to the 
density and temperature required for fusion. Plasma scientists have long used 
computer simulation to model this approach. However, the toroidal 
representation is complex, so for many years they have simplified it. First 
the torus is sliced and "bent" into a cylinder (of course,. doughnuts don't 
bend," but this is a mathematical doughnut, so it does whatever we want). 
Periodic boundary conditions are imposed across the ends of the cylinder and 
radial symmetry is assumed within it. The result is a two-dimensional, 
rectangular approximation to a "doughnut." This simplification is used 
because the computed parameters vary widely in space and time. State-of-the- 
art supercomputers can ha'ndle only crude approximations to the torus, whereas 
the simplification is tractable. To handle well the complexity of the torus 
requires computers 10 to 100 times faster than any available today. 


APPENDIX III 
PROGRESS IN TECHNOLOGY 

This appendix shows how progress in technology can be led by progress in 
computational capability. Appendixes I and II are also relevant to this 
issue. 


DESIGN OF NUCLEAR WEAPONS 

Since the beginning of the Manhattan project, computational models have played 
a central role in understanding and designing nuclear weapons. For many years 
ability to produce smaller and more efficient devices has been dependent on 
steady increases in computational power. The complexity of these designs has 
now reached a point where computational models have serious shortcomings. New 
developments in hydrodynamics and transport theory offer the possibility to 
correct this situation, but they require considerably more compute power than 
the techniques they replace. For example, implicit Monte Carlo technique? can 
require from 10 to 50 times the compute power of current techniques (3]. 
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PRODUCTION OF ENERGY FROM FUSION 

A second approach to production of energy from fusion involves inertial 
confinement. Construction of associated experimental facilities is expensive 
and time-consuming. In an effort to select designs that are most likely to 
succeed, computer simulation is used to study design alternatives. In some 
cases we are able to compute only a few percent of the phenomena of interest, 
even after running state-of-the-art supercomputers for as long as 100 hours. A 
major impact on fusion energy research could occur if we had new 
supercomputers that are 100 times faster than what we have today [A]. 



VLSI 

Colleagues in the semiconductor industry indicate that supercomputers may 
assume an important role in the design and simulation of VLSI devices. 
Confirmation should be obtained from experts in this area. 


APPENDIX IV 

FIFTH GENERATION COMPUTERS 

"Fifth generation" computers are computers designed to support application of 
artificial intelligence. These machines are also "super" in their level of 
performance, and like supercomputers necessitate new architectures. The 
Japanese are pursuing both areas and are spending about $100 million per year 
on the combined effort. Artificial intelligence has potential for broad 
application; for example, information analysis, intelligence based weaponry, 
and software engineering. Leadership in this area is essential to the 
security of the U.S. If the U.S. government initiates an R&D program in 
supercomputing, fifth generation capability should be included. 
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Organization 

I suggest that major funding for high performance computer research in 
universities be overseen by a Review Board, much in the style used by the 
British Science Research Council. Membership in the Board would be selected ™ 
from universities and from industry to represent competence in conceptual, 
analytical, manufacturing and application aspects of high performance computer 
systems. Such an arrangement is needed to avoid the haphazard nature of current 
funding which results from the different review and evaluation procedures used 
by different agencies, and from the difficulty of obtaining comparable reviews 
of competing projects. 

It might be best to have the evaluations of the Board advisory to the 
agencies so that funding decisions axe left in the agency's hands. 

Scope of the Area 

The area covered by the Review Board should be projects that involve funds 
for substantial equipment purchase and/or substantial use of non-academic re¬ 
search staff for pursuing relevant goals. Projects involving one faculty and 
one or two graduate students and no industrial cooperation should not come 
within the purview of the Board, although the relevant work of such groups 
should be brought to the attention of the Board. 




Dennis 


Goals 

The goal of research projects supported should be to further the under¬ 
standing of how concurrency may be exploited in future computer systems to 
achieve systems that have greatly improved performance (absolute), performance 
per unit cost, reliability/availability, and programmability. Systems designed 
to support computational physics, general-purpose computation, or signal 
processing, or data base processing stand equally to benefit from this organ!-' 
ration of research support. It would be an unfortunate error to restrict the 
purview of the Board to computer systems capable of supercomputer performance 
on computations of mathematical physics. 

An Opinion 

A small but growing group of experts in computer science research believes 
that future computer systems must be designed to provide a closer and better 
match of the structure of the hardware to the structure of programs to be run on 

the proposed system and the design of the language in which these programs are 
expressed. 

This aspect should be an important criterion in judging the suitability of 
projects for support. It is reasonable to expect that the feasibility of a com¬ 
plete system of software support be established - or even that it be implemented 
before major funding for hardware construction is committed. 


Methods of Evaluating Architectures 

Given a proposed computer systems architecture, it must pass certain tests 
of viability, one test (functionality) is to assure that programs expressed in 
an appropriate source language can, in fact, be translated into correct machine 
code. Simulation by means of programmed translators and interpreters is the 
standard method for doing this. 

The second test (performance) must evaluate the performance of a specific 
machine configuration for a suitable set of realistic application benchmark 
problems. There are four approaches to carrying this out: 
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- Build: A full or scaled down version of the proposed system can 
be built, scaled down problems run on it, and the results extrapo¬ 
lated to full size problems and systems. 

Emulate: Multiple hardware units or small microprogrammed machines 

can be built to perform the intended function of full-scale units. 
This is hard to distinguish from "Build" except that cruder imple¬ 
mentation techniques may be used to save on design time and commit¬ 
ment, and cost. This is distinct from simulation in that timing in¬ 
formation is not directly available, and performance levels must be 
ascertained from analysis of emulation results. 


Simulation: Simulation would most -likely be done using conventional 

techniques with traditional high-level languages and machines of con¬ 


ventional architecture. Higher performance may be achieved by 
running simulations on multi-processor systems, but true simulat 
is not easy to do on a multi-processor. 



Analysis: In some cases the applications have a sufficiently regulai 

structure and the mapping of problem onto machine level code is suf¬ 
ficiently understood that manual analysis is adequate to justify 
a claim of high performance. 


Evaluation is not complete without consideration of cost of fabrication for 
the proposed machine. Some proposals, which have significant merit otherwise, 
will fall down when judged by the cost of their construction. 



Gajski et al 
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Background 

Consider the goal of producing several types of broad-range, 
multi-gigaflop computer systems in the next decade. The basic problems 
to be solved are (as always) what hardware to use, what architectures to 
use, and what combination of software and algorithms to use in achieving 
the goals. We will avoid here the specifics of the details of performance' 
goals, exact application areas or breadth of application. But it is 
assumed that we are concerned with more-or-less general-purpose systems, 
at least general-purpose within an application area. Several general 
areas may be under consideration though. 

We will discuss some of the strengths and weaknesses of uni¬ 
versities proceeding alone and companies proceeding alone. This leads to 
the conclusion that some kind of joint effort is the best approach. 

Universities have traditionally been good at proposing 
architectural ideas, implementing research software projects, and carrying 
out theoretical and experimental performance evaluations. On the other 
hand, they have been poor at technology innovation. While the silicon 
foundry approach makes it possible for them to produce new chips, they will 
not be able to exploit the state of the art in performance or reliability. 
Another recent trend in academic computer architecture research seems to be 
toward the "one big idea" approach to computer architecture. This seems 
to produce architectures with a few simple, easy to understand ideas, that 
lead to a lot of related theoretical work, but too often without enough 
real-world applicability. 
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The focus in companies is shifting somewhat these days. The 
traditional "system houses" are continuing in some cases with products 
in which they have a long-term investment (e.g., Cray, CDC), others seem 
to have sharply retreated (e.g.. Burroughs), while some are trying to 
enter for the first time with new ideas (e.g., Denelcor). Some will 
attempt to copy the successes of others (e.g., the Japanese, Trilogy). 

An interesting shift of focus, however, is to the semiconductor houses 
themselves. Several are now producing interesting, innovative, low-speed 
VLSI products (e.g., the Intel 432). Several are breaking into the one 
megaflop (32-bit) chip or chip-set arena (e.g., TI and HP). 

It would seem that a great moment of opportunity is near in the 
possible wedding of high-speed architectures and VLSI components. In 
f a ct, we nave been approached by one semiconductor manufacturer with 
precisely the question, "How can we produce a chip set that OEM customers 
can fabricate into high performance multiprocessors of various types?" 

It is clear that companies are frequently forced to announce new products 
without enough time to think through the architectural, software, and 
applications ramifications of their work. Preliminary evaluations of the 
Intel 432 seem to be rather mixed, indicating a product that was perhaps 
rushed to the marketplace too quickly. In the area of high-speed systems, 
it is probably even more difficult to get a set of components that will 
perform well in a variety of systems and applications. 

A joint attack on the problem by industry/university cooperation 
would seem at this moment to have a high probability of success for several 
reasons. 

1) Several universities have well-developed efforts in 
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architecture, software, and algorithms for high-performance computers. 

2) Several companies have already produced high-performance 
component prototypes and are exploring the systems-level implications of 
these. 

3) The VHSIC program provides an example of a successful 
prototype program for this type of cooperation. 

As indicated above, the chances for success seem low if uni¬ 
versity and company efforts proceed independently. However, by properly 
managing a joint effort, some rather important breakthroughs should be 
possible. 


Goals 

The goal of the project would be to have one or more very 
high-performance computers built with advanced architectures and advanced 
technologies. Furthermore, these computers would be built by industrial 
organizations that would be likely to continue product-oriented development 
of the systems. 

We would argue that certain key conditions should be met to 
ensure the success of such a joint venture. It would almost certainly make 
sense to carry out two or three parallel projects which would be of different 
natures and which could cross-fertilize each other. Each such project should 
follow the outline below: 

1) Prototype . The architecture should be worked out and a (scaled 
down) prototype built using off-the-shelf parts in a short term (e.g., 
two-year) project. This milestone would include the development of 
register-transfer level diagrams together with timing diagrams that would 
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delivered to the industrial partner. By concentrating here on the 
architecture, the university partner would not be drawn into the fatal 
position of trying to push architecture and technology at the same time. 

2) Software and Performance Evaluation . In order to demonstrate 
the success of the project, software and algorithms should be produced by 
the university for the prototype so that it can be used on a variety of 
applications and its performance can be measured. This milestone is a 
validation of the prototype and simulation of the final product. These 
efforts should go hand-in-hand with the architecture design and not be 
afterthoughts (as we have seen in some university projects). 

2) Implementation . The initial design should have the property 
that it may be speeded up by (one or more of) several techniques. 

a) The design should be architecturally scalable, e.g., 
by replicating hardware parts it should perform faster. 

b) The design should be amenable to cost/performance improve¬ 
ment through hardware enhancements by the industrial partner. For example, 
the use of faster circuits or the use of custom chips instead of gate arrays 
would allow a faster clock. Custom VLSI chips would give a cost and size 
reduction as well. 

The software developed by the university partner on the prototype 
would be able to run on the final implementation. Further software develop¬ 
ment by the university in parallel with the final product implementation ~ 
should relieve the industrial partner of long software development delays. 


Summary 


A broad-based approach in which several industry-university teams 

were involved in separate projects would seem to be an ideal approach to 

reach the goal of multi-gigaflop systems in the 1980s. If the groups 

AI11-35 




Gajski et al 



were competing with respect to system speed, quality of results and 
schedule (not budget dollars), they would also be able to contribute to 
one another by exchanging ideas, serving on a joint advisory panel, etc. 

The recent DOE-sponsored meeting in New York demonstrated that 
there are a number of university groups working in the same general direction 
of high-performance computer system design. Some have narrower applicatidn 
goals than others, some have less software strength than others, and some 
have rather narrowly focused architectural ideas (i.e., the "one-issue" 
architectures). But, overall there is a lot of vitality in academic 
architecture research. 

In practice, most system manufacturers have a large investment in 
one or two architectural styles and are constantly seeking new hardware 
components and new software that will allow their architectural styles to 
gain a factor of two-to-four speed advantage over their competitors. The 
merchant semiconductor manufacturers have just recently arrived at this 


system level and are currently trying to determine what to do next. While 
their highest volume products are clearly in other areas, the diversity of 
32-bit products and projects they have revealed in the past year indicate 
that the time is exactly right to forge bonds between these three communities 
to accelerate and focus progress on superspeed systems for the 1980s. 
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Fred Ris 
IBM Research 
Yorktown Heights, NY 


The views expressed herein are strictly my personal views and should not 
be construed as being shared by other individuals or any organization, 
including my employer, IBM. 

We are here because we share a fundamental premise that there may be 
grounds for establishment of a national-interest activity in high-speed 
scientific computing systems for which the coordination (technical and 
financial) of the Federal government is indispensable. The end goal that 
seems the most appropriate is a computing system providing sustained (-not 
instantaneous peak) computing power two to three orders of magnitude in 
excess of that available today. General purpose computing systems 
delivering 10 to 100 gigaflops are extremely "unlikely to appear in the 
market in the 1990's without government intervention at this point. I 
therefore take the design and development of such a system to be the aim 
of the enterprise we postulate. 

Certain component technologies will not be accelerated any further by the 
existence of such a project and may therefore present potential 
unavoidable implementation constraints. While there will be some dispute 
as to precisely which component technologies fall into this category, I 
believe that high-speed circuit technology, levels of circuit 
integration, and capacity and access time of rotating storage media all 
have maximal impetus arising from the needs of commercial data processing. 
These technologies will therefore not be accelerated by the existence of a 
national project for high speed scientific computing, no matter how 
potent. Further, I do not expect these technologies to produce three 
orders of magnitude performance improvement in the next decade. 

If I assume that such a system can nevertheless be built in ten years, I 
would conclude that the technologies in which leverage must be applied are 
processor architecture and machine organization, system architecture, and 
application partitioning methodology. These are challenging research 
areas, of which many organizations are currently in pursuit. There is no 
particular national coherence in these pursuits, and it is therefore in 
those areas that a Federal initiative might make a substantial difference. 

Although I claim that silicon technology will not be significantly 
accelerated by scientific computing initiatives, it is nonetheless true 
that processor and system architecture and organization questions are not 
independent of this technology. The VHSIC program, for example, may be 
quite significant, and its progress must therefore be tracked to 
understand the implications of its results on the areas in which we will 
be interested. 
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Since there is no reason to believe we will be able to build large 
uniprocessors with sub-nanosecond cycle times in any known technology, 
multiprocessing will be inevitable. Most multiprocessing research 
presently focuses on communications among processors, and many of the 
communications network problems thus raised have lives of their own as 
interesting research topics. While understanding the fundamental pros 
and cons about particular network topologies is important, of greater 
importance for this endeavor is understanding the degree to which 
applications can be fit on multiprocessing implementations without 
compromising severely the domain of applicability of the implementation 
and without imposing inordinate human-intensive, front-end partitioning 
loads. 

To this end, I think it crucial to understand a few key representative 
applications for such computing systems, to understand whether the 
balance among computing power, memory capacity, backing store 
capacity/latency. and communications is reasonable, and also to provide 
the earliest possible indication that the technologies' courses of 
evolution will be consistent with the requirements of these applications; 
as they exist today Or as they can reasonably be projected to exist in a 
decade. 

A fundamental dilemma is hereby presented in that some of the applications 
that motivate such a national project in the first place are classified, 
particularly at the level of detail that is necessary for evaluating the 
suitability of prospective system configurations. In this case, knowing 
gross characteristics does not suffice; details of the inner workings of 
the applications are crucial for an understanding of how processing, 
memory, and communications requirements can be partitioned. 

One area that can absorb unlimited quantities of computing resources and 
that is fairly easy to grasp is atmospheric modeling. Weather models, 
while in some cases subject to constraints relating to academic 
proprietary interests, are for all intents and purposes freely accessible 
for the purposes of gaining the understanding suggested here. Some NASA 
applications may also have much the same properties. Classified national 
laboratories application requirements, while perhaps strongly motivating 
a Federal initiative, will not be of practical use in helping make design 
decisions about the systems that are ostensibly to be built for their 
benefit. 

It was repeatedly stressed at the June meeting that the probability of 
success of such an endeavor would be seriously compromised absent joint 
participation from the outset of government laboratories, universities, 
and industry. National laboratory involvement is required to provide 
application understanding and the ability to project it into the future, 
including knowing how today's implementations can be restructured if 
necessary to fit a different balance of component technological 
capabilities tomorrow. The universities are a source of invention and 
should also provide a mechanism for ensuring the integrity and consistency 
of experiments and the evaluation methodology of alternative 
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implementations. Industry must bridge this gap as well as bringing the 
realities of engineering and the ability to produce prototypes. 

These ingredients have been an integral part of the operation of the 
successful VLSI and PIPS projects sponsored by MITI, as well as in the 
ongoing Fifth Generatiop and High-Speed Scientific Computer projects. 
What the MITI development model adds to these components is a mechanism 
for achieving vital early consensus on technical direction. The Japanese 
government provides not only financing, but also technical leadership 
(often co-opted from national laboratories and universities) to achieve a 
nationally optimized balance between competition and co-operation. My 
concern is that without some similar mechanism in the U.S., there will be 
too much local (institutional) optimization, leading both to subcritical 
implementation mass for good ideas, and possibily precluding a more potent 
synthesis. A Federal government role is the obvious (but perhaps not the 
only) answer, which would be effective only if financial support is the 
mechanism for enforcing this discipline. 

I do not believe that an exact MITI model can work in the west because 
there are fundamental cultural forces in Japan that help to cement these 
relationships which are substantially weaker elsewhere. There are 
large-scale projects in the past twenty years that we might look to for 
retrospective insights about where organizational strengths and 
weaknesses lie. Among these are the Alaska pipeline, Apollo program, 
Concorde, the Space Shuttle program, and VHSIC. Similarly, we may want to 
contemplate the direct initiatives the French are taking in 
semiconductors and personal computing. Finally, we should consider 
private-sector initiatives recently launched in the U.S.; notably the 
Semiconductor Research Cooperative of the Semiconductor Industry 
Association and the proposed Microelectronics and Computer Technology 
Enterprises. 

In summary, while there are plenty of exciting technical challenges ahead 
of us, I believe the political and social problems will be equally hard 
and will appear on the critical path sooner rather than later. 
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September 9, 1982 

Dr. Ettore F. Infante 
National Science Foundation 
1800 G Street, NW 
Room 339 

Washington, DC 20550 


Dear Dr. Infante: 

The draft .report on Large Scale Computing in Science and Engineering 
is fairly good, but fails to make what I believe to be the essential point. 

We do not lag the Japanese in general access or applications know how 
for supercomputers. We are falling behind in the plans to design and 
actually construct the supercomputers for the 1990's. Fujitsu and Hitachi 
have already announced their supercomputers for delivery in 1983 which 
are on the average superior to the Cray 2 (not only the X-MP) and possibl^^ 
the next CDC computer. Besides the Japanese government is supporting 
component research and construction of a next generation (for the 1990 s). 

We must stress not only the "long-range research basic to the design 
of a new supercomputer systems", but we must also stress that the 
government should procure by competitive bid as early as possible a 100 BFlof 
machine—hopefully before 1990. 

Without this aim the paper is worthless. It just asks for more money 
to do what is already being done and which will not match the Japanese 
competition. We need a real goal to make this a new exciting venture for 
the government to support as well as to provide the U.S. with the continued 
leadership it may lose (most certainly will) in this area. 


Sincerely, 





Sidney Fernbach 


SF: kw 



AIII-40 


Fernbach 



development. Government committees are not good for anything but 
offering criticism. They should be used in an advisory capacity. They 
may have to sort out alternatives and be prepared to make compromises, 
but no changes. 


Sincerely, 



Sidney Fernbach 

SF: kw 
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October 7, 1982 

Dr. Ettore F. Infante 
National Science Foundation 
1800 G Street, NW 
Room 339 

Washington, DC 20550 

Subject: Comments on the report from Group 3 of the Lax Committee. 

Dear Dr. Infante: 


Distinguishing between the various ongoing university efforts in 
"Advanced" computer architecture is like distinguishing between Illiac's, 
Maniac's, and Johnniac's in the 1950's. Most efforts are very similar, 
the chief differences among them being in how to switch from one unit to 
another. One is led to think that putting a large number of small processors 
together is the only way to create a super-scale computing engine. 


There should be no question that this "research" should be continued 
but does it bear still greater expenditures? They should be pushed to 
completion, if possible. There are some techniques that are still so 
far from reality that a great deal more work is necessary (such concepts 
as data flow and the systolic array, for example). This work should be 
supported at higher levels and also pushed into reality. 



More "Traditional" systems with special features, such as vector 
boxes or functional processors can be exploited now. Let us set our goals 
and go. If we start researching the parallel processor approach it will 
be years before we start moving. Even with this architecture we can move 
fast if we take advantage of what has already been done, e.g. Denelcor's 
HEP, CDC’s AFP, and LLNL's S-l. 


It is interesting that all the talk is about a CPU-memory system. 
Little or no mention is made of the essential peripheral equipment that 
can help make or break the system. Not only do we need a 100 GFLOP 
machine, but we also need matching peripherals. We must define our goals 
and let the manufacturers tell us how to get there. They will if there 
is any incentive. 

Our immediate need is a target performance and target date. The cost 
may be high but we can reach both goals if we pay the price. Certainly 
there should be a cooperative effort between industry, government and 
university. Only one, however, should be in charge. This role should 
be played by industry—the manufacturer which takes the lead in the 
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Scisncs, Encinssrir.c 


In January 1982, a Federal Coordinating 
and Tecnr.ciccy (FCCSIT) Panel on Supercomputers was formed to exarr.me what, 
if anytr.ing, one 2.3. Government should do to stimulate the use and 
development of supercomputers. As a result of the FCCSFT panel discussions, 
three interagency working groups were formed. These groups are to examine 


w 1C _ u ± __ 


ic surercommuter issues: 


Wnat snou.c tne Government dc to ensure that tne 2.3. retains its leaf 
in supercomputers? This croup is named the Procurement Group. 

What snculc the Government do to make supercomputers available to more 


researchers, particularly in universities? This 
Access Grouo. 


P—^ »-s r^amsc cnG 


2. A third group was formed to provide coordination among Gover-nmen: 
. agencies funding research ccr.tributinc to tne technc.ocy base. 

■S 

This report descri'oes the findings anc reccrmer.daticns of tne Access 
Tne Access Group consists or tne oo-lowmc members: 


Robert Kann, 202 
Kennetn Walcrer., NASA 
P.icnarc Cunlavey, COG 
George Rogers, DC! 


The Access Group was concerned cmy with making pclicv recommendations 
on access for scientists and engineers to supercomputers. Supercomputers 
are tne most powerful computers available at any given time for dome 
scientific calculations. Future Fifth 'Generation machines for artificial 
intelligence applications were not considered. The Group relied upon 
previous studies for documentation that a significant need for access exists 
and it did net attempt to quantitatively assess, the access requirements ir. 
the short time available. Further, the Grcuo did not examine research a^d 
development needs for supercompu 11 no, n.cr did it oor.sider otr.er r.tes of 
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individual iabcratories or oy single programs. Very few researor.ers, 
particularly in universities, nave access to these- .machines and, as a 
result, are unable to actress many forefront researe.n prcciems. 

The need fcr mere super computing m me C.S. researcr. community cannot 
be met simply by wider access to existing federally funded computer 
resources m government laboratories. Many federally funded facilities 
are operating at or near full capacity, cr plan to use availacie capacity 
fcr future programmeric growth. Facilities that now support research in 
weapons developrer.t cr nuclear energy, fcr example, 'would have to be 
totally restructured fcr unclassified users to share computing resources. 
The three university supercomputer centers are underutilized. These 
universities have had difficulty m obtaining sufficient resources to 
provide the necessary, broad, user friendly services. Commercial 
service bureaus do nave available computer time to sell. 

The programmatic need -to provide access to researchers geographically 
dispersed across the country, -and economics dictate serving users 
remotely by comnur.i cat ions networks from a limited number of 
supercomputer centers. 


High quality service wnich maximizes the productivity of researchers 
should be a prime consideration when establishing supercomputer 
services for remote users. Several years will be required to obtain 
high quality service from a newly established center. 

There are rwc federally supported supercomputer centers that have a 
relatively long history cf providing good service to large numbers of 
remote users: the Magnetic Fusion Energy !MFE' Computer Center at tne 
Lawrence Livermore National Laboratory, anc the National Center for 
Atmospheric Research (NCAR) Computer Center in Boulder, Colorado. 

There are insufficient opportunities for training students in the 
physical sciences and engineering in the use cf supercomputers and for 
training computer science students in the operation of supercomputer 
systems. 
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The Government should taKe action to provide access to supercomputers 
for government funded researcr.ers :r. government laocratories, universities, 
anc ctner institutions wnc nave large computational propieir.s requiring me 
use of supercomputers. Supercomputers are critical in solving the complex 
oroclens vital to our seouritv, economic, and scientific"interest. There is 
clear evidence tn.ar me i.ac.% go access to suoeroom.puters limits progress m 
manv asoects of science enc engineering. 


The Working Groups's recommendations were developed to help solve this 
national prcrlem. Recommendations are made or. near-term apprtaor.es to 
providing access to large numoers of users quickly and also for the longer 
term. The Group tried to identify mose features of the MFE ar.c iJCA-P. 
operations (described in Appendix A) which have mace them, successful in 
providing nigh quality services to remote users. Recognition of these, 
features was a key factor in developing many of the recommendations. 


A. continuing interagency group snoeuc oe cesignatec to provice 
coordination among agencies to help ensure appropriate sharing of 
resources, to avoid duplication of effort anc to maintain a dialog on policy 
anc tscr.mcai issues. 

To meet immediate needs, each federal agency' supporting scientific anc 
engineering research should: 

1. if not already accomplished, assess the number of supercomputers 
"required to meet the realistic needs of the cotimunity each agency 

supports both for scientific research and graduate education;. 

2. develop plans to ensure that high quality service is provided to their 
' research communities to meet the needs; 
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4. initiate access by corrmor. carriers ror ccnrmur.i cat ions by remote users. 
In order to provide initial service, 960C baud lines should be made 
available to researchers. 


To meet long-term ooyectives, eacn agency should: 


coordinate the development of plans for scientific networks with cone: 
agencies. Over the next several years, the goal should re to design 
and implement an integrated system of networks linking researc.ners wi: 
each other and with computing centers. 


6. establisr. new supercomputer centers and associated networks as nested. 
Seme cr the considerations for managing and siting centers 
described in this report may be of use to the various agencies. 

Existing centers shouic be used to help train personnel for new centers 
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r supercomputer centers to maintain compete: 
r.ign quality service. The most practical 
oier.t fluctuations m funding is for agencies 
,c to meet anticioatec center costs. 
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1C. support graduate student training in the use of supercomputers anc as 
appropriate in the operation of supercomputer centers. 

Anc, finally, it is recommended that: 


11. federal agencies should continuously upgrade the capacity anc capabilities 
of federally supported superconpcter centers; and 

12. a mechanism should be established to trade computer time between 
agencies. The objective cf such tracing would be to permit a researcher 
who has contracts with more than one agency to use only one computer 
system, thus saving tome and effort. 

REPORT OF THE :CCSIT WORKING GROUP 
ON ACCESS TO SUPERCOMPUTERS 


Present Status of Scientific Concur me 


Backcrour.d - The Issue 


During the decade of the 1970's, the government laoorator:es rapidly 
began to acquire supercomputers, the most advanced general purpose 
computer system available. These supercomputers were used as powerful tools 
for conducting research in fusion energy, weapon systems, aerodynamics, 
anc atmospheric science. They were the Control Data 76C0, Burroughs Illiac 
IV, and Texas Instrument Advanced Scientific Computer. They were noted for 
tneir greatly increased speed and memory over previous generations cf 
computers. n 












substantialiy oc-osted the capacilities of- supercomputer: 
z»re anc mere researcners to use tne tools for r.umerica. 
processing of very large data sets. Today, fully ccr.fi 
systems with memories of at least 1 megawerd anc speeds 
MF10?S exist at a number cf government laboratories and 
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In the Federal Government, supercomputers are used primarily for weatr.er 
forecasting, space, energy, nuclear weapons, anc science research project 
This is summarized below: 


Department of Energy . The primary use of supercomputers 
in" DOE laboratories*has been for modeling nuclear weapons 
performance. Modeling has substantially reduced the number 
of experimental tests'required to finalize design of new 
weapons. Historically, the DOE weapons laboratories have been 
the world’s largest user of supercomputers and they have been 
instrumental in developing software to utilize nev; generations 
of machines. 

In more recent years, applications of supercomputers have 
increased in various energy technologies inducing fusion 
research and nuclear energy. 

** 

The energy sciences programs (High Energy anc Nuclear Physics, 
Basic Energy Sciences, and Health and Environmental Researco¬ 
al sc" have large complex scientific problems anc are just 
beginning to benefit substantially from the use of 
supercomputers. 

National Science Foundation . Many university researcners 
have found that in order to understand important aspects of 
physical, biological, and chemical events and processes, 
numerical simulation on supercomputers is critically needed. 

In recent years, NSF has provided supercomputer facilities 
for atmospheric and ocean sciences at NCA?.. But only small 
numbers of university scientists ana engineers in ether 
disciplines have had access to supercomputers at other 
national laboritories, and only if their research was relevent 
to the laboratory mission. Much of the university based work 
in numerical modeling has been accomplished on lew cost mini¬ 
computer ‘Systems*■. ■ ■ ! v ^ 
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communications, anc comcucatic: 
important mcrecients in t.nis or op ran. Witnm tne space 
science data management environment, supercomputers are one 
of tne essential elements. Tne steacy increase m cats 
vol'ume, variety of space—derives oata, subsequent processing 
requirements, and modeling neecs c: scientiric mvestigaticns 
have greatly increased the need for tne fastest and most 
caoadle computers. NASA's researe.n anc tecrmcicgy pat.ntinoing 
efforts m tne computationally based sciences, particularly 
computational fluid dynamics, has put the L’.S. m the 
leacersn^o role in several ciscipiines. These rorerront 
researe.n efforts recuire access to tne leaning edge 
computational capabilities. As a result of continued 
improvements lh numerical algor items as wei- as dramatic 
increases in processing speed and storage capacity of recent 
suDercomouters, computational fluid dynamics has Deccme a 
cowerfui and widely usee toe- m the aerodynamic design or 
aircraft and aerospace systems. 

Department of Ccnmerce . Supercomputers are being used 
extensively m the weather forecasting and meteorological 
research activities of the Department's National Oceanic and 
Atmcspnenc Administration. The dynamic nature of the 
atmosphere anc its effect on r.umar. activity has led to tne 
development of complex technologies m weatner ooservmg ■ 
anc 

processing 
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laws tr.at govern tne 
sophisticated numerical 
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research is heavily dependent on supercomputers to compute tne 
complex interactive orocesses with tne extensive space and time 
scales involved, and the large data input required to establish 
initial conditions. 




Coucied with this federal activity, industry has successfully used 
supercomputers in a number of applications including: exploration for 
petroleum, and mineral deposits; maximizing petroleum recovery; design 
simulation of aircraft components; design and simu_ation of complex 
mtecratec circuits; design of future supercomputers; chemical process 
control anc optimization; combustion, modeling for internal combustion 
encir.es; and computer graphics for image generation and animation. 
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Ir. spite of this steady growth :r. super coma: ■ 
availability to seme research groups, owe ve: 
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Large numbers of researchers 
students, etc.; do not nave 


and potential researchers (graduate 
ffactive access to supercomputers; anc 


The capaoilities of today's supercomputers 
magnitude toe small for attaching many cur 
problems. 


are several orders of 
er.t anc urgent research 


This critical situation was identified oy tne 
in Science and Engineering, c.nairsc by Peter D 
anc DOD (with the cooperation cf TOE and NASA) 
panel tc explore the problems, needs, and eppe 
computing. In the December 1982 report of thi 
stated that: 


Panel on Large Scale Computing 
. Lax. In June cf 1982 the NSF 
sponsored a workshop for this 
rtunities in large scale 
s panel (Lax Report; it was 


"The U.S. has been, anc continues to be, the leader in supercomputer 
technology and in the use of supercomputers m science and engineering. 
The present position of leadership is evidenced by the dominance of 
the supercomputer market by American producers and by the successful 
exploitation of supercomputing at national laboratories. However, the 
Panel finds that this position of leadership is seriously undermined by 
the lack of broad scale exploitation, outside of a few national 
laboratories, of the scientific and engineering opportunities offered 
by supercomputing, anc by a slowdown in the introduction of new 
generations of supercomputers. This threat becomes real in light cf 
the mager thrust in advanced supercomputer design that is being 
mounted by the Japanese Government and industry, and by vigorous 
governmental programs in the United Kingdom, West Germany, France, and 
Japan to make supercomputers available and easily accessible to their 
research anc technological communities.-- 


As a resu.t of tnis assessment, tne Lax panel rscccmendec the estao 
cf a National Program on Large Scale Computing, This program sheui; 
participation a nc cooperation by federal agencies, universities, an: 
industry in carrying out four goals: 


ishmer.t 

foster 


increase access for the research community tc supercomputers; 

increase research in tne computational tools (software, algorithms, 
etc.) needed for effective use of supercomputers; 

train personnel in use cf large scale computing; and 

stimulate R&D for substantially increased capability and capacity in 
the next several generations' of supercomputers. 


i/see page 5 cf the Repor 


Science and Engineering, 
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(December 1982) 
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In addition to the Lax report, there are a nuttier of other relevant 
some of which are listed m Appendix 5. 


Several of the European nations nave moved actressively to supply 
researoners witr access to supercomputers. .0 is not uncomnc.. —r 
resear oner s who nave difficulty retaining supercomputer time m tr.e 
country to do super computing cn European macnin.es curing summer vis 
sabbaticals, etc. 


its, 


A list of countries and macn.mes entirety or partial-y avai.aoxe rt 
university, researchers is provided below. This list may not be com 
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Country 

S ite 

Commuter 

Germany 

Max Planck 

CRAY-1 


3ochum 

University cf 

CYBER-2C5 


Karlsruhe 
University of 

CYSER-2C5 


Stuttgart 

CRAY-1 


KFA 

CRAY-1 

France 

Ecole 


- 

Poiytecnnique 

CRAY-1 

England 

Daresbury 
University of 

CRAY-1 


London 

University of 

CRAY-1 

* 

Manchester 
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Sweden. 

Volvo 

Cray-1 

Japan. 

University cf 



Tokyo 

HITACHI S 
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In Japan, supercomputer companies are placing their first 
machines in university centers. 


orocuctic; 


In December 1982 the Science Advisor, Dr. Keywcrth, initiated a review of 
U.S. policy in the field of supercomputers by the Federal Coordinating 
Council cn Science Engineering and Technology, FCCSET. As part of the 
review, this interagency working croup was established to consider the 
question cf research access to supercomputers and to make recommendations 
the FCCSET. Two parallel working groups were established to review the 
other asoects cf the general topic. This report examines the access issue 









Durmc its cei liberations, 
cessation of one Xacr.eoir 


one Access Group neard presenoaoicns or. one 
Fusion. Energy Computer server.--, and one National. 
Center for Atmospheric Researcr.'S Computer Network. These owe r.etvcr.-'.s 
provide superce.T.puoer access oc one largest rumors cf remote users or any 
existing supercomputer centers. The Group also heard agency plans for 
providing increased access to supercomputers in the future. These 
presentations included: a report by NSF working group or. plans to provide 
significantly enhanced opportunities for one academic researor. ccrrr.unity to 
access supercomputers; NASA's plans to provide more access to its 
contractors as part of the Numerical Aerodynamic Simulation program; 

XC's plans for the ARFAnet and the wideband packet satellite s-ystem; 

and DOS's plans to expand access to the Magnetic Fusion Energy MFE) net fo: 

contractors funded by the Department's Energy Research, programs. 

Two members cf the group, L. Lee and J. Decker, together with K. Curtis 
and A. Thaler of NSF, and S. Karin of GA Technologies (serving as a 
technical consultant) visited supercomputer centers at the University of 
Minnesota, NCAR, MASA-Ames, and the Lawrence Livermore National Laboratory 
(MFE). Discussions were also held with a number of scientists vne are 
remete users cf the NCAR center. 


TT 


CaDabiiitv ana Service Requirements of Supercomputer Centers 


High Quality Service 

It has been well documented that supercomputers are critical :r. attacking 
manv difficult scientific anc tecr.nclogical problems. The remote use of 
supercomputers should be made convenient and friendly to the user. 

Otherwise the inefficiencies at the user end can oe surstar.tial, making tr.e 
real cost cf computing very high. In fact, researchers may net use or.is 
important scientific tool if its use preves difficult to learn, frustrating 
and generally inconvenient. The Access Group heard anecdotal evidence cf 
poor "supercomputer service causing researchers to return to work or, tneir 
local minicomputers. For these reasons, tne Group wishes to stress the 
importance of providing high quality service. 

As mentioned "earlier , several members of this FCCSET access working group 
visited the University of Minnesota, NCAR, NASA-Ames, and MFE. The purpose 
of the trip was to gain an understanding of tne factors that make for 
successful supercomputer centers. On this trip, there were several 
copertur.ities to discuss those factors with users. The group concluded t.-.a 
the following features are considered essential for providing users with 
hich cualitv supercomputer resources. ^ 






medium speed 4800-9622 ups: terminal access and high speed '56 
Kbps or greater; file transfer capability far access cy rerr.cte 
users: 

trillion hit cr greater cn-line mass storage capacity; 

full-time user consultants on software, hardware, and 
ccnmun 1 ca 11 ons ; 

extensive system documentation and software utilities and libraries 
state-cf-tne-art graphics tools; 
around the ciccx operations; and 
interactive operating systems. 


Ccrr.outer Networks 


A men cuaiity, hign bandwidth computer network is one of the most important 
requirements for providing effective supercomputer service to remote users. 
Networxs provide an important service that goes far beyond the simple 
concept of providing supercomputer access to remote users. Geographically 
dispersed researchers can collaborate daily using electronic mail and shared 
program, anc data files to work together effectively on the same problem. _ 
Duplication of programtinc effort can be virtually eliminated if 
cc.laocratcrs can easily snare resources. The cost of the communications 
networx m use on the NMFFCC is a small fraction of the total operating 
budget, yet in terms of the valuable time spent by a scientist using this 
rac:_:ry, mgn bancwictr. response is a crucial cart of the service. The 
1220 cps terminal access service commonly available today is not adequate 
for one .arcs majority cf supercomputer users. A minimum, of 9602 ops is 
necessary to provide the level of response available on local minicomputers; 
there is no reason why this level of response snculc not be available on 
supercomputers. 


For file transfers of data and programs between local computers and 
supercomputer centers, a minimum of 56 kbps is usable, but 1.5 Tbps 
is both desirable and feasible. 3ecause the efficiency cf the researchers 
using the centers is by far the most important consideration, and because 
the cost of communications is a small fraction cf the total cost of 
providing these services, the highest quality network available snculd oe 
obtained. 








SDeciai Characteristics cf Sucerccrr.ccter Centers 


Supercomputer centers are defined here, to -ear. centers equipped j,itn. sr.e :r 
mere Class VI ccmputers; an adequate merarcr.ica. mass storage system; 
•state-or -tne-art output faci-ities; a ru_. ccrr,p_enent or ays ter. s , 
ooeraticr.s anc user services start; anc network support tor rer.ote users 
throughout the V.3. Two such centers, NMFECC and NCAR, are supported cy 
government researc.o programs to provide high qua.rty service tc a large 
number of researchers in universities, incustry, and ^aocratcrles. i.r. cot.. 
cases, it required several years to build up competent staff; to assemcie 
complete supercomputer systems, and tc ceveiop operating systems anc 
coimuni cat ions software. In tne case or the 'CtF:—a sigr.ir-car.t per— n. 
cf the original staff were recruited from, tne LLNL central computer center, 
a classified center serving a large number of internal users. Even with 
the experienced initial staff, it tcck several years tc develop good serv.c 
to remote users. 

In terms cf skilled staff, system nareware and sortware, anc communications 
facilities, tne requirements for the operation of a successful center 
serving a laroe number of remote users are quite different from these for a 
center operating in a batch mode for on-site anc dial-up users. Although 
it is certainly possible to expand some cf the small supercomputer centers 
to provide high cuaiity service tc a large number of renote users,' it “ J '' 
require a significant amount of time for system development anc hirin 
training cf competent staff. 
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Some effort or. software development could be saved by duplicating parts of 
an established system such as the iMFECC. However, some cf the components 
usee in these centers are now no longer manufactured anc new nareware and 
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rtwars development will be requirec to replace tnese components. 


Supercomputer venders have not provided adequate software :n tne past. 

DOE laocratcries, the largest users cf supercomputers, have developed their 
own operating systems to provide a hig.n quality, interactive computing 
environment. New supercomputer centers acquiring new classes cf 
supercomputers may either have to develop the capability cf writing an 
operating system or wait until anotner center has developed one. This will 
be required* unless the vendors ccnmt resources to develop satisfactory 
interactive operating systems. 

As indicated above, operation cf a supercomputer center to provide high 
quality service to remote users is net a simple step up from operation cf a 
cenerai ouroose mainframe that comes with gooc^scftware support anc a 
complete*set of peripherals from large conmercial venders. In order to 
reduce the learning time for any new supercomputer center, there should be 
a mechanism for the transfer of knowledge from existing centers. One 
possibility is to send staff from a new center to existing centers for 
training. 






The cost of operating gooc super: 
reasonably well established. Al 
of the service provided and wim 
NCAR and M-tFECC can be used as :: 
budget of the NCAR. center is app 
two Cray-1' s .with a staff of 12? 
total value of installed equipme 


computer' center service for remote users is' 
though costs will depend on the exact nature 
the institution providing the service, both 
nc:cations of costs. The annual operating 
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The total annual ooerating cost of the ^ECC is approximately $12,200,200. 
This center operates two Cray-1’s and a CCC “588, anc serves about 2200 users 
The total value cf the installed equipment is S23-2CM. An approximate 
breakdown cf the NMFECC budget for FY 19S3 follows: 


Manpower (80 FTEs) 

S 6,400,088 

Lease (1 Cray) 

-5 -7 nn rx^n 

C t / U Xj f OO 1 j 

Maintenance 

i inn 

Ccmnunications 

800,300 

Supplies and Service 

900,000 

Miscellaneous 

400,080 

Total 

$12,502,C8C 


IV, Seme Conclusions on Operating anc Sitmc of Supercomputer Centers 

The list cf siting considerations provided below is net to be taxer, as ail 
inclusive, nor is it intended to be a strict set cf criteria tr.at agencies 
should follow-. The list does represent some observations anc tnougr.ts of 
the working Group which should be carefully considered. Many of the 
considerations cane from examining existing successful operations. 

(1) The proper philosophy of operation of a supercomputer center is 

absolutely essential for providing good services to a large collection 
'of remote users. Such a center has tc c*v£ ens hignest priority ec 
satisfying user needs. In particular, the desire to experiment with 
the system in a research mode is incompatible with serving users. The 
service mission must be compatible with the goals of the institution th 
rranaaes the center. ^ 


(21 The center must receive frequent, organized input from users. Both 
NCAR and MFE have advisory committees which are credited with 
contributing substantially tc the success of those centers. 
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Stacie funding is an important ingredient of success. Computer centers 
nave many fixed operating costs such as equipment leases and 
maintenance, staff salaries, conr.unications, utilities, supplies, -etc. 
There are two primary options for meeting tnese costs: direct cnarges 
tc users or direct funding of one center. In the former case, computer 
time can ce purcnased 'ey researc.ners who nave the money, and cnccse to 
dc sc. In the latter case, computer time is allotted 'ey one funding 
acencv or a peer review committee acting on oenalf of tne agency. 

Direct charges to users does net guarantee stacie funding. In fact, if 
crciected revenues are not being met, tne center has to increase the 
hourly rate. This may drive customers away, further reducing revenues. 
Note that a center serving remote users is in a mere precarious position 
than a center cr.iv serving users within an institution. In the latter 
case, the management of the institution can obtain a ccrrrr* i tiusn w z, t om 
researchers tc spend fanes st the local center anc avoid a disastrous 
financial situation. The two centers with the longest reccro of 
success in serving remote users, NCAR anc MFE, both receive direct 
funding. 


In the case wnere an agency funds a computer center directly at the 
beginning of the fiscal year, computer time is allocated by the agency 
in a manner similar to program funds. When a principal investigator 
submits a research proposal, he may request resources consisting of 
supercomputer time as well as operating funds. Requests for computer 
time are reviewed in the same manner as operating funds. The normal- 
situation is that requests for computer time like those for program 
funds far exceed the totals available and consequently both have to be 
manaoec carefull’v as scarce resources. 


4) 


( 5 ) 


There is merit in locating one*or more supercomputer centers near or in 
universities to serve educational needs. However, since supercomputer 
centers are costly to establish and operate, it is olear that only a 
verv few renters are possible. It tnus seems appropriate that 
university consortia might be formed to operate centers in several regions 
of the country. Further, tne consortia cculc establish or hire management 
organizations who have as their primary objective providing computer 
service. The two centers with the longest record of successors both 
bv university contractors but in an indirect fashion. is run oy 


run 


the University of California through the Lawrence Livermore National 
Laboratory and NCAR is managed and operated by the University 
Corporation for Atmospheric Research—a consortium of 49 universities.- 

There is a considerable cost saving realized by locating several 
supercomputers at one center compared with establishing several small 
centers. The incremental cost of adding one more computer to a center 
is generally small compared to establishing a, new center. One center 
can economically serve a large geographical area since cornnunications 
costs are a relatively small fraction of the total operating cost. For 
tne MFC net, the communications costs are a small fraction of the 
total. Furthermore, supercomputer centers once established should 
be maintained at the state-of-the-art, which means replacing mainframes 
every five or six years. It is questionable whether funds will be 
available in the future tc keep a large number of centers up tc date. 








While the economics cf smal- supercomputer centers, e.g. me 
supercomputer, are less favcracie man ror _arcer centers, there is 
slso ar. ucoer limr to commuter center size. It is net c-ear wnat 
determines tnat licit. Some pcssicie ractcrs are: number or 
computers, size cf staff, r.umoer or users, r.umcer or cisc.piines 
served, number of Government ceparrments- programs servec, etc. .c 
fate, successful Government funded supercomputer centers nave served 
smc.e mission crocrams an.c.cr smg_e oiscip-inss. 

6. Location of suoercommuter centers m a gocc science ar.c engineering 
environment wnere correcting is cone routine-’/ is a car mite plus. 

Such n.earov activities help cor.outer center starr maintain cirect 
contact with the problems or meir users. 

,7; Since suoercomouters represent leading—eege tecnr.cuogy, a supercomputer 
center must have the capability to test and implement new software and 
hardware concepts. 

8'' A center musv provide ccoc user documentation, train me, anc full-time 
user assistance. 

.9) center must oe able tc attract high quality, personnel to carry out 
rranv of the items listed above. 


V. Gcals for f J.S. Policy 

Supercomputers are very powerful scientific tools that nave applications m 
many areas of research" and in some fields have become essential for advancing 
the" state-of-the-art. In order„that the U.S. retain its techr.clogical 
leacersr.it; m these fields, supercomputers must be made availaole to the 
menv researeners in university and government laboratories across the 
cour.tr/ who currently do not have access. Toward tms end, me goals cf 
I'.S. Government policy snculc oe: 


?c provide access to supercomputers 
researen suoocrtec ov or or interes: 
such cacao ill tv; anc 


;r researc.ners wnc are concucting 
:o the oovernm.er.t and who recuire 


To aid the education of university students in the use 
suoe r ccrr.cc te r s. 




The implementation 


pcl;oy based on 


oosi. w:._ oe mstrumentaj 
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:r. many areas of science anc 
one ccunorv with the use of a 


Ensuring ohat tne J. S. maintains its lead ir. one development of 
advanced supercomouter systems by expanding one applications, anc m 
turn, tne marker for supercomputers m one U.S.; and 


(c! Provicinc the ocporturnties for training scientists and engineers m 
the use of supercomputers oc meet future needs for experienced 
researchers in Government programs and industry. 


VI. Reccirmenca 1 1 ons 

The Government should take action to provide access to supercomputers 
for government funded researchers in government laooratories, universities, 
anc ether institutions who have large computational problems requiring the 
use of supercomputers. Supercomputers are critical in solving the complex 
problems vital to our security, economic and scientific interest. There is 
clear evidence that the lack of access to supercomputers limits progress in 
many aspects of science ana engineering. 


The Working Groups's recommendations were developed to help solve this 
national.prooiem. Recommendations are made on near-term approaches for 
providing access to large numcers'of users in the shortest time and also for 
the longer term. The Group tried to identify those features of tne MET and 
;vCAR operations (described m Appendix A) which have made 
them successful* in providing high quality services to remote users. 
Recognition of those features was a key factor m developing many of the 
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Recorrmanfeations to meet Immediate Needs 

A continuing interagency group should be designated to provide 
coordination among agencies to help ensure appropriate sharing of resources, 
to avoid duplication of effort, and to maintain a dialog on policy end 
technical issues. 

1. The number of supercomputers required to meet tne needs of each agency 
should be assessed if suer, an assessment has net already been 
conducted. 


Each 
serv 
requ 
C0O i 


federal agency should develop plans 
ice is provided to its own researcr 
ires good communications and well-ma 
oated to service. 


to ensure that high quality 
ommunity. High quality servio 
.aged supercomputer centers 




In order to acccrr.pl:sn me accve recommendation, some of me existing 
supercomputer centers already providing seal:ty service to r=ncce users 

provide service to a wider ccrrrr.mity m me near 
, T.oe Zepar~=r.: cf Energy is taxing tms accroach tv 
network to serve me energy sciences programs. 

In order to initiate access as soon as possible, existing 
communications sneuid be usee as a start, including common carriers, 
ARPAnet, MFEnet, etc. In order to provide initial service, %0C baud 
lines sneuid oe made available to researchers. 


Recommendations to meet Long-Term, Objectives 

Federal agencies should coordinate the development cf plans for 
scientific networks. Over me next several years, the goal should be 
to design ar.c implement an integrated system cf networks linking 
researchers with each other anc with computing centers. 

Establishment cf new supercomputer centers and associated networks 
should be initiated fcv individual agencies as needed to meet their 
long-term objectives. The benefits derived from adding supercomputer 
centers include opportunities to improve education anc training and to 
optimize the coupling of researchers to supercomputers. Same of the 
considerations for siting centers described in this report may be of 
use to the various agencies, ilt must be recognized that it will likely 
take several years before a new center is providing high quality service. 
The use cf existing centers to tram personnel for new centers would 
aid development of new centers substantially. 


should oe used to 
expanding me MFE 


For a center with two super' 
case cf S25-33M, annual ope 
Because the total costs of 
incremental costs cf adding 
nuncer cf centers should be 
trained anc needs develop. 


orr.purers and a total installed ecuicment 
sting costs are likely to oe S12-S15M per 
carting a new center are r.igner than the 
a machine to an existing center, the tota 
limited ar.c sneuid increase cniv as staff 


vea: 
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Staole funding of supercomputer centers is an essential ingredient in 
maintaining competent personnel anc for providing high-quality service. 

The most practical means for realizing truly staole funding is by *- • 
agencies providing funcmc to meet center costs. The agencies 
would then allocate computer time in the same manner as they allocate 
program funds. This approach has ceer. usee successfully by ns? JNCAR) 
and DCE (MFZ:. Eirect charging of users particularly when they are remote 
will not provide the necessary stability of revenues. 


High bandwidth communications will undoubtedly play an important role 
in supercomputer networks in the future for applications such as 
interactive grannies. The federal agencies should begin experimenting 
with' high bancwicths communications systems. 









Each agency sncuid support graduate student trair.ir. 
super computers. Fellowship programs and summer ccl 
established. Special allocations of supercomputer 
available for prefects that involve graduate stucer. 
centers should oe encouraged to employ students or. 
full-time basis wherever appropriate to provide pra 
the management, operation, ^ahd systems 'software fern. 


g m the use or 
loquia snould oe 
time sncuid oe made 
ts. S upe r ccrr.pu ter 
a part-time or 
c~ v cal =xoer:ence in 
ancs of orovidmc 


supercomputer serv 


ces. 


Additional Recommendations 


Federal agencies should continuously update the capacity and capaoilities 
of federally supported supercomputer centers. Supercomputers are by 
definition tne most powerful scientific computers available at any¬ 
time . This implies that a supercomputer center must have tne latest 
state-cf-the-art machine to meet the evolving computational requirements 
of the research community. 

A mecnanism snould be estaolished to trade computer time between 
agencies. The objective cf such tracing would be to permit a researcner 
who has contracts with more than one agency to use only one computer 
system, thus saving time anc effort. 
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ail of these elements are essential, 
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Comnuter Networks 



In tne late ISoO's the Defense Advanced Research Projects Agency (DARPA) 
began an international packet switched computer network called tne ARPAnet. 
Today that network hosts over 200 computers at nearly 100 universities, 
government laooratories, and private research firms. The backbone Oi uhe 
network is 50,000 bits per second- land line augmented by satellite links to 
Hawaii and Europe. ARPAnet has been split into an operational network, 
called MILNET, and an experimental .network, called ARPANET. 

The ARPANET is the only operational network that provides the full range of 
network services (virtual terminal access, file transfer, electronic mail 
and bulletin boards, and internetworking protocols) for a large variety of 
host computers. This serves as an example of the Kina of services desirable 
for a future operational network for supercomputers. 


\ second examnle is a wideband racket satellite network wnicn was developed 
with DARPA suooort to explore'issues in high-speed computer communications 
and real-time data transmission including integrated data and voice. The 
_ £ r v o r cterates across the continental U.S. using a single broadcast channel 
on the Western Union WE STAR satellite. The system operates codec or uncoded 
at cata rates up to 3.OSS M bits/sec. 



A third example is the satellite network run by the NMPECC. This provides 
dual 5okbps channels from 4.S kbps and 9.6 kbps access lanes into the 
regional centers. The direct satellite link, provides reliable, medium band 
width foie transfer capability as well as terminal access. 


Commercial va 
to i22G bps), 
from the ARPA 
independently 
as SSr's CSNZ 
ARPANz.- , tr.e 
communities a 


lue added networks provide only low-speed terminal access (up 
The packet switched networks, TELENET and UNINET, evolved 
NIT. TYMNET, a circuit switched network, evolved 
-, but also rrovides only terminal access. Other networks, sue 
;T, Bell Labs' USENET, and EDUCOM's EDUNET, are built on the 
commercial networks, and dial-up service. These serve various 
end terminal access and electronic mail. 


h 




Computer networks provide an i.T.pcr rant service zr.az goes rar oeycr.c me 
"simoie function cf allowing remote users terminal access tc superccmrxiters. 
Networks allow geographically dispersed researchers tc collaborate 
effectively by sharing software, tracing ideas, and ever, jointly ’writing 
research caDers. 


National Macnetic rosier. Energy lcit.ccting 


For almost ten years now, the Mr I computer center nas oeen providing fusion 
enerev researchers with large scale computing services. Scientists are 
developing several categories cf computer simulations tc study me physics c: 
magnetically confined plasmas, while engineers are designing the models 
needed m fusion reactor design studies. The MFE provides high level 
supercomputer capability to about 2000 users at more than 3C remote 
locations. At the heart of the center is a CEC 7600 plus two CRAY 
suoercomputers with one and two mi.lion worcs or memory, respectively. The 
file management system has three levels of service - disk systems, mass 
store tape cartridges, and automatic tape library. 


The cornerstone of the MFE- system is the network. Four mager TOE fusion 
enemy laboratories are linked by dual 56 kbps satellite links tc MFE. Many 
universities anc other researen laboratories are connected to these centers by 
medium speed land lines (4800 - 960C bps). Other users, not linked by one 
cf the above, may dial up the center through TYMNET, ARPANET, direct 
commercial dial,'or FTS. This nationwide network is efficient, cost- 
effective, and allows remote fusion researchers to access the center with 
several levels cf local computing capability. 


There are nurrerous aspects of the MFE center which contribute to its success 
in serving fusion energy researchers. Seme of these are enumerated below: 


Growing national network consisting 
speed anc hardware capabilities; 


c_ owCTmur‘.icaticn 


- Fully interactive time-snaring system on tne 


AY computers; 


- Consultants 
engineering 


to users who specialize m mathematics, physics, and 
software anc who have experience in hardware; and 


Capability cf developing new services such as an interactive 
multiprocessor operating system fer supercomputers, new graphics 
tccis, anc an advanced interactive query system. 





■ National Center for Atmcscnenc Researcr. 'SCAR- 

The National Center for Atmcspneric Research r.as been providing 
computational services to the atmcspneric science comer.ity for twenty years 
now. The accuisition of successive generations or computers r.as made it 
oossibie for the science to address prcclems m the tnecry or weather 
credicticn, climate studies, severe stems research, ocean circulations, anc 
the influences of one sun, land., and oceans or. me atmespnere. Today, NCAR 
provices computational resources to more than 900 scientists at “5 
institutions. The hardware at )X1A?. includes rwc CRAY computers !1 mecaworc 
menorv, 64 bit word lengths), two IBM 4341 interactive computers, and a mass 
storage system on wnich are stored 7.6 trillion bits or oata. In accition 
to NCAR's" important role in serving the computational needs of the 
atmospheric science community, iXZAR has become a leading computational 
resource for the ocean science com unity. 

One of the most significant strengths in the management and operation of the 
NCAR computing facility is the involvement of the user community. SCAR is 
managed by the University Corporation for Atmospheric Research (UCAR1 - a 
consortium of 49 universities. The atmospheric anc oceanic research 
communities (NCAP. ar.d university scientists) participate in the allocation 
of computing resources, setting of plans and priorities, and reviewing of 
policies anc operating procedures. This has led to the development of the 
following very extensive and useful services: 

- On-line access to many large meteorological data sets 

- Large consultant group and educational program 

- Multiple capabilities in graphical displays 
Numerous numerical and utility software libraries 


There is a great need for a cost effective, eiricier.t, ar.c .ow-errer rate 
communication system. Much cf the computing cr. the NCAR system is data 
intensive. NCAR has recently taken some steps to improve access for its 
remote users. As a result of an NCAR agreement with the national 
corrrr.uni ca t l'cns network UN I NET, 951 of the researchers at universities can 
consnur.icate with the NCAR system at 1203 baud and at the cost cf a local 
teleohcne call. Plans in the future include the development cf a high 
bandwidth communication system linking university researchers to NCAR. 





Lcs Alamos National laboratory 

One cf the largest scientific c cm cuter centers is at Lcs Alamos National 
Laboratory. Currently, the laboratory serves 4000 users m Los A lanes anc 
1000 remote researchers. The primary function of the computer facility is 
to serve the computational needs of one nuclear weapons program.. However, 
Los Alamos also supports the computational needs of researchers m nuclear 
anc plasma physics; inorganic, nuclear, and physical chemistry; mathematics 
anc computer science; metallurgy;'biomedical effects of radiation; and earth 
anc space sciences. The nucleus of the computing facility is a collection 
of interactive computers and high capacity, rapid access storage devices. 
These include; 


- 5 GRAYS 

- 4 CDC 7600s 


- Numerous CYBER 802's anc VAX 11/760s 

- IBM 3852 Mass Storage System 

This network of computers anc data storage devices is partitioned into three 
security partitions: secure, administrative, and open. 

The Los Alamos Computer Center performs research, development, and support 
in areas of computer science and engineering. A very high-speed interactive 
display system cf both graphical and textural data is being developed. 

Also, the center supports five operating systems, all of which provide the 
user with interactive computing. Recently a Denelccr HIP-1 multiprocessor 
computer was acquired fer experimentation. The center is studying several 
hign-level languages to determine their usefulness in solving a variety of 
scientific problems. In addition to providing users with many direct 
services, suer, as consulting, troublesnootmg, assisting with equipment, anc 
writing, maintaining, and enhancing a variety of lioraries, the tenter is 
developing a users' education program centered around olasses, seminars, 
computer-assisted instruction, and videotapes. 

The Computing Division at Los Alamos recently began to provide computing 
support to over 400 external users associated with the Defense Nuclear 
Agency (DN'A). It also supports some supercomputer access from university 
research groups. Remote access to Los Alamos is supported through 
conrc erciai dial-up lines, TELENET, ARFANET, and Los Alamos' own extended 
network (XNET). VAX computers with the DEC VMS operating system are 
required for using XNET. Also, six 56 kbps lines have been implemented to 
serve remote DNA users. 







N’ASA-Ames Researtn Center 

MASA is planning to develop, at its Ames Research Center, a Numerical 
Aerodynamic Simulation ;MAS; system. This is a unique supercomputer 


solution of aerocvr.amic anc f I 
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iierr.s. 


1985, tne NAS will‘become operational and provide tnese sucercommuter svstem 
services to NASA, CCD, industry, other government agencies, anc 
•universities. 

Ir. mid-1982 NASA significantly changed its approach for the development of 
tne NAS. The former apprcac.o was for NASA to procure a contract for the 
development of a very high performance supercomputer system specificailv 
designed for solving viscous, three-dimensional, fluid-flow equations. 

In the latter phases of this development, neitner of the two competitive 
vendors could assure NASA that they could meet their requirements on 
performance. Under the new NAS approach, Ames will acquire in FY 1984 a 
prototype of a Class VII large-scale computer. This should be a 
250 megaflop, 64 megaword main memory system. This would be followed 
by s second procurement ir. FY 1987 of a 1 gigaficp (1002 necaflops; and 256 
megawcrd Class VII computer. The NAS program calls for a systematic anc 
evolutionary approach to acquiring computer-system technologies to 
maintain a leading-edge performance capability. Hence, for examcie, users 
will be accessing one supercomputer system while NASA works on procuring and 
installing a significantly better performing follow-on system. 

There is a need for NASA to provide the full end-to-end capabilities. This 
will span the ranee from mass storage systems to microprocessor work 
stations, and will provide lancline and satellite data communication links 
to offsite users. 

The overall cost of the NAS in the FY 1983-58 time frame will still be in 
tr.e neignbcrhccc of S122M. However, m tnis new incremental apprcacr. to 
■developing tr.e NAS, users will have access much earlier, FY 19S4 versus 
FY 19SS. One interesting and appealing aspect of this clan is tr.at mere is 
a good possibility tr.at as the NAS evolves, it will become a multi-vendor 
complex of supercomputer systems. This may prove to be very advar.taoecus to 
a diverse aerodynamic industrv user base. 


NASA-Ames is currently operating a CRAY-iS and a CDC 7600 for the internal 
needs of the research center. This fall the center will replace tne CRAY-IS 
with a CRAY-XMP. A study is bemc initiated on the needs of long haul 
communications for remote user groups of the NAS. 
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Universities 

Several Class VI computers have recently been acquired at ma^or universities 
- a CYBER 2G5 at Colorado State University ;C3U) anc Purdue University and 
a CRAY-i at the University of Minnesota (UK). At least two of these systems 
are seriously ’underutilized. After mere-than one- year of availability the 
Colorado State system, it is being used at approximately 32% of its capacity 
The University of Minnesota has upgraded their CRAY from 1/2 million words t 
one million. Ail three universities have received seme financial assistance 
from vendors. These university centers are striving tc increase staff 
cXp6i icHcs m servmc isrce nurroers of r erne re re see rimers ir.d te csvslcc 
meir ccrnr.cr.i cat ions svstems zzz rerrete users. 


Ccrtrercial Services 










Additional Supercomputer Studies 

The report on Comturational Facilities for Theoretical Physics 
(June 1961 recommended mat a national computationa. pnysics 
network oe estaoiisr.ee witr. a super corn pc ter raci _ icy as a 
orincioai node anc a set of recior.a_ anc *ocai nodes, eacr. witr 
its own commuting capacity. 


The 1982 NAS report entitled An Assessment of Computational 
Resources Required for Ocean Circulation Modeling surveyed the 
ocean modeling conrr.cnl-ty and found that tne protectee computing 
resources of their academic cun nunicy will fall short of their 
needs in 1984 by a factor of more than two. As a result of this 
report, NCAR nas been selected to take the responsibility for NSI 
supported oceanographic computing. 


The Atmospheric Sciences Advanced Vector Commuter Justif 
document (December 1982) stated a need for increased com 
resources at NCAR by a factor of 5 over their presently 
Cray's. 
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The computational fluid dynamics community has concluded in the 
NRC Report on Computational Fluid Dynamics (1982) that tnev neec a 
1000 MFLGF computer with a 502 million wore memory in order to 
simulate the flow field about a complete aircraft using the 
Reynolds-averaged Navier Stokes equations. 


In the NRC P.eccrt cn Data Ma: 
are a r.umoer of reccrmer.es ti 
science cats. Among them, i 
closely with the scier.mific 
computational capacilities, 
software. 
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In January 1982 the TOE report Magnetic Fusion Zr. 
panel of researchers recommenced that the MFE acq 
several years additional supercomputer systems ;C 
Class VII) and that the communication systems be 
adding more dedicated lines and increasing bandwi 


erov ar.c Commuter 
;u:re m tne next 
:lass VI and a 
upgraded by 
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The NSF report A National Car.auting Environment for Academic 
Research recommends that up to 10 new supercomputer centers be 
established tc serve the documented needs of the researon 
community. 
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